W8-W9. Similar Matrices, Complex Vectors and Matrices, Hermitian and Unitary Matrices

1. Summary

1.1 Similar Matrices

When we diagonalize a matrix \(A\) as \(A = PDP^{-1}\), we are expressing \(A\) in terms of a simpler, diagonal matrix \(D\). The relationship \(D = P^{-1}AP\) is a special case of a far more general concept: similarity. Two matrices are similar when they represent the same linear transformation but written in different coordinate systems.

Definition (Similar Matrices): Two \(n \times n\) matrices \(A\) and \(B\) are called similar if there exists an invertible matrix \(P\) such that:

\[B = P^{-1}AP\]

We write \(A \sim B\). The matrix \(P\) is called the change of basis matrix — it converts coordinates from one basis to another.

Geometric Interpretation: If \(T\) is a linear transformation and \(A\), \(B\) are its matrix representations in two different bases, then \(A\) and \(B\) are similar. The matrix \(P\) converts between the two coordinate systems. This is why similar matrices “look different” but share the same underlying geometry.

1.1.1 Invariants Under Similarity

Similar matrices share a set of fundamental properties that remain unchanged under the change of basis. These are called invariants of the similarity relation.

Theorem: If \(B = P^{-1}AP\), then:

1. Same Determinant: \(\det(B) = \det(A)\)

   Proof: \(\det(B) = \det(P^{-1}AP) = \det(P^{-1})\det(A)\det(P) = \dfrac{1}{\det(P)}\cdot\det(A)\cdot\det(P) = \det(A)\)

2. Same Trace: \(\text{tr}(B) = \text{tr}(A)\)

   Proof: Using the cyclic property \(\text{tr}(XYZ) = \text{tr}(ZXY)\): \(\text{tr}(P^{-1}AP) = \text{tr}(APP^{-1}) = \text{tr}(A)\)

3. Same Characteristic Polynomial: \(\det(B - \lambda I) = \det(A - \lambda I)\)

   Proof: \(\det(P^{-1}AP - \lambda I) = \det(P^{-1}(A - \lambda I)P) = \det(A - \lambda I)\)

4. Same Eigenvalues (with same algebraic multiplicities): Since the characteristic polynomials are equal, their roots — the eigenvalues — are identical.

Important Note: Having the same eigenvalues does not guarantee similarity. For example, \(A = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}\) and \(B = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}\) both have \(\lambda = 1\) with multiplicity 2. But \(A = I\) is only similar to itself, so \(A \not\sim B\). Similarity requires more: equal eigenvalues plus equal geometric multiplicities for each one.

1.1.2 Powers of Similar Matrices

If \(B = P^{-1}AP\), then \(A^n\) is similar to \(B^n\) for any positive integer \(n\):

\[B^n = (P^{-1}AP)^n = P^{-1}A^nP\]

Proof sketch: \((P^{-1}AP)(P^{-1}AP) = P^{-1}A(PP^{-1})AP = P^{-1}A^2P\). Repeat by induction.

Key application: If \(A\) is diagonalizable, \(A = PDP^{-1}\), then: \[A^k = PD^kP^{-1}, \quad D^k = \text{diag}(\lambda_1^k,\, \lambda_2^k,\, \ldots,\, \lambda_n^k)\]

This makes computing high powers of a matrix practical — instead of multiplying \(A\) by itself \(k\) times, we just raise each eigenvalue to the \(k\)-th power.

1.2 The Complex Field

Real matrices can have complex eigenvalues (for example, rotation matrices). To work with these fully and guarantee that every matrix has eigenvalues, we extend from \(\mathbb{R}\) to the complex field \(\mathbb{C}\).

Definition: A complex number has the form \(z = a + bi\), where:

• \(a = \text{Re}(z)\) is the real part
• \(b = \text{Im}(z)\) is the imaginary part
• \(i\) is the imaginary unit, satisfying \(i^2 = -1\)

Operations:

• Addition: \((a + bi) + (c + di) = (a + c) + (b + d)i\)
• Multiplication: \((a + bi)(c + di) = (ac - bd) + (ad + bc)i\)
• Conjugate: \(\bar{z} = a - bi\) (flip the sign of the imaginary part)
• Modulus: \(|z| = \sqrt{a^2 + b^2}\) (the “length” of \(z\))
• Division: \(\dfrac{z}{w} = \dfrac{z\bar{w}}{|w|^2}\) (multiply numerator and denominator by \(\bar{w}\))

Polar Form: Every complex number can be written as: \[z = r\,e^{i\theta} = r(\cos\theta + i\sin\theta)\] where \(r = |z|\) is the modulus and \(\theta = \arg(z)\) is the argument. This follows from Euler’s formula: \(e^{i\theta} = \cos\theta + i\sin\theta\).

Powers become trivial in polar form: \(z^n = r^n e^{in\theta}\).

Key Properties of the Complex Conjugate:

• \(z + \bar{z} = 2\,\text{Re}(z)\) (always real)
• \(z - \bar{z} = 2i\,\text{Im}(z)\) (always purely imaginary)
• \(z\bar{z} = |z|^2\) (always a non-negative real number — this is the crucial property)
• \(\overline{z_1 + z_2} = \bar{z}_1 + \bar{z}_2\) and \(\overline{z_1 z_2} = \bar{z}_1\,\bar{z}_2\)

Why the complex field is essential for linear algebra: The Fundamental Theorem of Algebra states that over \(\mathbb{C}\), every degree-\(n\) polynomial has exactly \(n\) roots (counted with multiplicity). This means every \(n \times n\) complex matrix has exactly \(n\) eigenvalues (counted with multiplicity). In contrast, a real matrix such as the \(90°\) rotation \(\begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}\) has no real eigenvalues at all — but over \(\mathbb{C}\) it has eigenvalues \(\lambda = \pm i\).

1.3 Complex Vectors in \(\mathbb{C}^n\)

We now work with vectors whose entries are complex numbers.

Definition: A complex vector \(\mathbf{v} \in \mathbb{C}^n\) is an ordered \(n\)-tuple of complex numbers: \[\mathbf{v} = \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix}, \quad v_i \in \mathbb{C}\]

Basic operations (addition and scalar multiplication with \(\alpha \in \mathbb{C}\)) work component-wise exactly as in \(\mathbb{R}^n\).

1.3.1 The Hermitian Inner Product

In real vector spaces, the inner product is \(\langle \mathbf{x}, \mathbf{y} \rangle = \mathbf{x}^T\mathbf{y} = \sum x_i y_i\). Can we use the same definition over \(\mathbb{C}\)?

No — and here is why. Consider \(\mathbf{x} = (1, i)^T \in \mathbb{C}^2\). The naive formula gives: \[\sum x_i x_i = 1\cdot 1 + i\cdot i = 1 + i^2 = 0\]

A nonzero vector would have “zero length” — completely nonsensical. The problem is that \(i^2 = -1\) allows cancellation of positive terms.

The fix: use the complex conjugate on one argument. Conjugating turns \(i^2 = -1\) into \(|i|^2 = i\bar{i} = 1\), which is always non-negative.

Definition (Hermitian Inner Product): For \(\mathbf{x}, \mathbf{y} \in \mathbb{C}^n\): \[\langle \mathbf{x}, \mathbf{y} \rangle = \mathbf{x}^H\mathbf{y} = \sum_{i=1}^n x_i\,\overline{y_i}\]

where \(\mathbf{x}^H = \bar{\mathbf{x}}^T\) is the conjugate transpose of \(\mathbf{x}\).

Note: Some textbooks (and the physics community) conjugate the first argument instead: \(\langle \mathbf{x}, \mathbf{y} \rangle = \sum \bar{x}_i y_i\). Both conventions are valid; just be consistent. Here we follow the convention used in the lecture.

Why this works: Now \(\langle \mathbf{x}, \mathbf{x} \rangle = \sum x_i\overline{x_i} = \sum |x_i|^2 \geq 0\), and it equals zero only when all \(x_i = 0\).

Key Properties of the Hermitian Inner Product:

1. Conjugate Symmetry: \(\langle \mathbf{x}, \mathbf{y} \rangle = \overline{\langle \mathbf{y}, \mathbf{x} \rangle}\) (not simply \(\langle \mathbf{y}, \mathbf{x} \rangle\) as in the real case)
2. Linearity in the first argument: \(\langle \alpha\mathbf{x}, \mathbf{y} \rangle = \alpha\langle \mathbf{x}, \mathbf{y} \rangle\)
3. Conjugate linearity in the second argument: \(\langle \mathbf{x}, \alpha\mathbf{y} \rangle = \bar{\alpha}\langle \mathbf{x}, \mathbf{y} \rangle\)
4. Positive Definiteness: \(\langle \mathbf{x}, \mathbf{x} \rangle \geq 0\), with equality if and only if \(\mathbf{x} = \mathbf{0}\)

Why the inner product must be sesquilinear (conjugate-linear), not bilinear:

There is a second geometric reason why conjugation is unavoidable. A natural requirement for any inner product is that scaling a vector by a complex scalar \(\alpha\) should scale its length by \(|\alpha|\) (the modulus, not \(\alpha\) itself): \[\|\alpha\mathbf{x}\|^2 = \langle \alpha\mathbf{x}, \alpha\mathbf{x} \rangle = |\alpha|^2\langle \mathbf{x}, \mathbf{x} \rangle\]

For this to hold, the inner product cannot be bilinear — if it were, \(\langle \alpha\mathbf{x}, \alpha\mathbf{x} \rangle = \alpha^2\langle \mathbf{x}, \mathbf{x} \rangle\), and \(|\alpha^2| \neq |\alpha|^2\) in general (e.g. for \(\alpha = i\), \(\alpha^2 = -1\) while \(|\alpha|^2 = 1\)). The fix is to make the product conjugate-linear in one argument, so that \(\langle \alpha\mathbf{x}, \alpha\mathbf{y} \rangle = \alpha\bar{\alpha}\langle \mathbf{x}, \mathbf{y} \rangle = |\alpha|^2\langle \mathbf{x}, \mathbf{y} \rangle\). This is why the Hermitian inner product is called sesquilinear (“one-and-a-half linear”): linear in one argument, conjugate-linear in the other.

The simplest case — \(\mathbb{C}\) as a one-dimensional vector space:

Even for a single complex number, conjugation is the only natural choice. Define \(\langle z, w \rangle = z\bar{w}\) for \(z, w \in \mathbb{C}\). Then \(\langle z, z \rangle = z\bar{z} = |z|^2\), which is real and non-negative. The alternative without conjugation, \(\langle z, w \rangle = zw\), would give \(\langle z, z \rangle = z^2\), which is not necessarily real (e.g. \(z = i\) gives \(z^2 = -1 < 0\)) and cannot represent a squared length.

Summary — what conjugation guarantees:

• \(\langle \mathbf{x}, \mathbf{x} \rangle\) is real and positive for \(\mathbf{x} \neq \mathbf{0}\) (positive definiteness).
• The induced norm scales correctly: \(\|\alpha\mathbf{x}\| = |\alpha|\|\mathbf{x}\|\) for all \(\alpha \in \mathbb{C}\).
• The inner product aligns with the natural modulus on \(\mathbb{C}\): \(\langle z, z \rangle = |z|^2\).

Without conjugation, the structure would be a bilinear form — useful in some algebraic contexts, but lacking the positive-definite metric properties required for a genuine inner product space.

1.3.2 Norm and Orthogonality in \(\mathbb{C}^n\)

Definition (Euclidean Norm): The norm (length) of \(\mathbf{v} \in \mathbb{C}^n\) is: \[\|\mathbf{v}\| = \sqrt{\langle \mathbf{v}, \mathbf{v} \rangle} = \sqrt{\sum_{i=1}^n |v_i|^2}\]

For a real vector this reduces to the familiar \(\sqrt{\sum v_i^2}\).

Definition (Orthogonality): Two vectors \(\mathbf{x}, \mathbf{y} \in \mathbb{C}^n\) are orthogonal if \(\langle \mathbf{x}, \mathbf{y} \rangle = 0\).

Cauchy-Schwarz Inequality: For all \(\mathbf{x}, \mathbf{y} \in \mathbb{C}^n\): \[|\langle \mathbf{x}, \mathbf{y} \rangle| \leq \|\mathbf{x}\|\cdot\|\mathbf{y}\|\]

Gram-Schmidt orthogonalization works in \(\mathbb{C}^n\) using the Hermitian inner product in place of the dot product.

1.4 Complex Matrices

A complex matrix \(A \in \mathbb{C}^{m \times n}\) is a matrix with entries \(a_{ij} \in \mathbb{C}\). All standard matrix operations (addition, scalar multiplication, matrix multiplication) work exactly as for real matrices.

1.4.1 The Conjugate Transpose (Hermitian Adjoint)

The most important new operation for complex matrices is the conjugate transpose, also called the Hermitian adjoint.

Definition: For \(A \in \mathbb{C}^{m \times n}\), the conjugate transpose \(A^H \in \mathbb{C}^{n \times m}\) (also written \(A^*\)) is defined entry-by-entry as: \[(A^H)_{ij} = \overline{A_{ji}}\]

In words: first transpose (swap rows and columns), then conjugate every entry.

For a real matrix, \(A^H = A^T\) (conjugating real numbers changes nothing).

Example: Let \(A = \begin{bmatrix} 1 & i & 2 \\ 1-i & 0 & 3i \end{bmatrix}\). Then: \[A^H = \begin{bmatrix} 1 & \overline{1-i} \\ \overline{i} & 0 \\ \overline{2} & \overline{3i} \end{bmatrix} = \begin{bmatrix} 1 & 1+i \\ -i & 0 \\ 2 & -3i \end{bmatrix}\]

1.4.2 Properties of the Conjugate Transpose

For matrices \(A\), \(B\) of appropriate sizes and \(\alpha \in \mathbb{C}\):

• \((A^H)^H = A\)
• \((A + B)^H = A^H + B^H\)
• \((\alpha A)^H = \bar{\alpha}\,A^H\)
• \((AB)^H = B^H A^H\) (reversal rule — identical to transpose)
• \((A^{-1})^H = (A^H)^{-1}\) (if \(A\) is invertible)
• \(\langle A\mathbf{x},\,\mathbf{y} \rangle = \langle \mathbf{x},\,A^H\mathbf{y} \rangle\) (key inner product identity)

The last property is fundamental: it says \(A^H\) is the unique matrix that “moves” from the left to the right of an inner product. This is why \(A^H\) is called the adjoint of \(A\).

1.5 Hermitian Matrices

The complex analog of a real symmetric matrix (\(A = A^T\)) is the Hermitian matrix.

Definition (Hermitian Matrix): A square complex matrix \(A \in \mathbb{C}^{n \times n}\) is called Hermitian (or self-adjoint) if: \[A = A^H\]

Entry-level condition: \(a_{ij} = \overline{a_{ji}}\) for all \(i,j\). Consequences:

• Diagonal entries are real: \(a_{ii} = \overline{a_{ii}} \Rightarrow a_{ii} \in \mathbb{R}\)
• Off-diagonal entries come in conjugate pairs: \(a_{12} = \overline{a_{21}}\), etc.

Every real symmetric matrix is a special case of a Hermitian matrix (with all imaginary parts zero).

Example: \[H = \begin{bmatrix} 1 & 2 + i & 3 \\ 2 - i & 4 & i \\ 3 & -i & 0 \end{bmatrix}\]

Check: diagonal entries 1, 4, 0 are real; \(h_{12} = 2+i = \overline{h_{21}} = \overline{2-i}\) ✓.

1.5.1 Spectral Theorem for Hermitian Matrices

Hermitian matrices have three remarkable eigenvalue properties that collectively form the Spectral Theorem:

Theorem 1 — All eigenvalues of a Hermitian matrix are real.

Proof: Let \(A\mathbf{v} = \lambda\mathbf{v}\) with \(\mathbf{v} \neq \mathbf{0}\). Compute \(\langle \mathbf{v}, A\mathbf{v} \rangle\) two ways:

\[\langle \mathbf{v}, A\mathbf{v} \rangle = \langle \mathbf{v}, \lambda\mathbf{v} \rangle = \lambda\|\mathbf{v}\|^2\]

\[\langle \mathbf{v}, A\mathbf{v} \rangle = \langle A^H\mathbf{v}, \mathbf{v} \rangle = \langle A\mathbf{v}, \mathbf{v} \rangle = \overline{\langle \mathbf{v}, A\mathbf{v} \rangle} = \bar{\lambda}\|\mathbf{v}\|^2\]

Since \(\|\mathbf{v}\|^2 > 0\): \(\lambda = \bar{\lambda}\), which means \(\lambda \in \mathbb{R}\). \(\square\)

Theorem 2 — Eigenvectors for distinct eigenvalues of a Hermitian matrix are orthogonal.

Proof: Let \(A\mathbf{v} = \lambda\mathbf{v}\), \(A\mathbf{w} = \mu\mathbf{w}\), \(\lambda \neq \mu\) (both real). Compute \(\langle A\mathbf{v}, \mathbf{w} \rangle\) two ways:

\[\langle A\mathbf{v}, \mathbf{w} \rangle = \langle \lambda\mathbf{v}, \mathbf{w} \rangle = \bar{\lambda}\langle \mathbf{v}, \mathbf{w} \rangle = \lambda\langle \mathbf{v}, \mathbf{w} \rangle \quad (\text{since }\lambda\in\mathbb{R})\]

\[\langle A\mathbf{v}, \mathbf{w} \rangle = \langle \mathbf{v}, A^H\mathbf{w} \rangle = \langle \mathbf{v}, A\mathbf{w} \rangle = \mu\langle \mathbf{v}, \mathbf{w} \rangle\]

Equating: \((\lambda - \mu)\langle \mathbf{v}, \mathbf{w} \rangle = 0\). Since \(\lambda \neq \mu\), we get \(\langle \mathbf{v}, \mathbf{w} \rangle = 0\). \(\square\)

Theorem 3 (Spectral Theorem) — Every Hermitian matrix is unitarily diagonalizable.

There exists a unitary matrix \(U\) (defined in §1.6) and a real diagonal matrix \(\Lambda\) such that: \[A = U\Lambda U^H\]

The diagonal entries of \(\Lambda\) are the (real) eigenvalues of \(A\), and the columns of \(U\) are orthonormal eigenvectors.

Proof (by induction on \(n\)):

Base case: \(n = 1\). A \(1\times1\) Hermitian matrix is a real number \(a \in \mathbb{R}\). Take \(U = [1]\) and \(\Lambda = [a]\): trivially \(A = U\Lambda U^H\). \(\square\)

Inductive step: Assume the theorem holds for all Hermitian matrices of size \((n{-}1)\times(n{-}1)\). Let \(A \in \mathbb{C}^{n\times n}\) be Hermitian.

1. Get a unit eigenvector. By Theorem 1, \(A\) has at least one real eigenvalue \(\lambda_1\). Choose a corresponding unit eigenvector \(\mathbf{u}_1\) (\(\|\mathbf{u}_1\| = 1\)).

2. Build a first unitary. Extend \(\{\mathbf{u}_1\}\) to an orthonormal basis \(\{\mathbf{u}_1, \mathbf{u}_2, \ldots, \mathbf{u}_n\}\) of \(\mathbb{C}^n\) via Gram-Schmidt. Let \(U_1 = [\mathbf{u}_1 \; \mathbf{u}_2 \; \cdots \; \mathbf{u}_n]\) (a unitary matrix).

3. Block-diagonalize. Compute the \((j,1)\) entry of \(U_1^* A U_1\): \[(U_1^* A U_1)_{j1} = \mathbf{u}_j^* A\mathbf{u}_1 = \mathbf{u}_j^*(\lambda_1\mathbf{u}_1) = \lambda_1\langle \mathbf{u}_j, \mathbf{u}_1\rangle = \lambda_1\delta_{j1}\] So the first column of \(U_1^* A U_1\) is \((\lambda_1, 0, \ldots, 0)^T\), and since \(U_1^* A U_1\) is Hermitian, the first row is \((\lambda_1, 0, \ldots, 0)\) as well: \[U_1^* A U_1 = \begin{pmatrix} \lambda_1 & 0 & \cdots & 0 \\ 0 & & & \\ \vdots & & A_{n-1} & \\ 0 & & & \end{pmatrix}\] where \(A_{n-1}\) is an \((n{-}1)\times(n{-}1)\) Hermitian matrix.

4. Apply the induction hypothesis. By the induction hypothesis, there exists a unitary \(V\) of size \((n{-}1)\times(n{-}1)\) such that \(V^* A_{n-1} V = \text{diag}(\lambda_2, \ldots, \lambda_n)\). Build the block-diagonal unitary: \[U_2 = \begin{pmatrix} 1 & 0 \\ 0 & V \end{pmatrix}\]

5. Combine. Set \(U = U_1 U_2\) (product of unitaries is unitary). Then: \[U^* A U = U_2^*(U_1^* A U_1)U_2 = \begin{pmatrix} 1 & 0 \\ 0 & V^* \end{pmatrix}\begin{pmatrix} \lambda_1 & 0 \\ 0 & A_{n-1} \end{pmatrix}\begin{pmatrix} 1 & 0 \\ 0 & V \end{pmatrix} = \begin{pmatrix} \lambda_1 & 0 \\ 0 & V^* A_{n-1} V \end{pmatrix} = \text{diag}(\lambda_1, \ldots, \lambda_n) = \Lambda\]

Thus \(A = U\Lambda U^*\), completing the induction. \(\square\)

1.6 Unitary Matrices

The complex analog of a real orthogonal matrix (\(Q^T = Q^{-1}\)) is the unitary matrix.

Definition (Unitary Matrix): A square matrix \(U \in \mathbb{C}^{n \times n}\) is called unitary if: \[U^H U = U U^H = I \quad \Longleftrightarrow \quad U^{-1} = U^H\]

Structure: The columns of \(U\) form an orthonormal basis for \(\mathbb{C}^n\) with respect to the Hermitian inner product: \[\langle \mathbf{u}_i, \mathbf{u}_j \rangle = \delta_{ij}\]

Equivalently, the rows of \(U\) also form an orthonormal basis.

Example: \(U = \dfrac{1}{\sqrt{2}}\begin{bmatrix} 1 & i \\ i & 1 \end{bmatrix}\) is unitary. Check:

\[U^H = \frac{1}{\sqrt{2}}\begin{bmatrix} 1 & -i \\ -i & 1 \end{bmatrix}, \quad U^H U = \frac{1}{2}\begin{bmatrix} 1+1 & -i+i \\ -i+i & 1+1 \end{bmatrix} = I \quad \checkmark\]

1.6.1 Properties of Unitary Matrices

Property 1 (Preserves inner product): \(\langle U\mathbf{x}, U\mathbf{y} \rangle = \langle \mathbf{x}, \mathbf{y} \rangle\)

Proof: \((U\mathbf{x})^H(U\mathbf{y}) = \mathbf{x}^H U^H U \mathbf{y} = \mathbf{x}^H I \mathbf{y} = \langle \mathbf{x}, \mathbf{y} \rangle\). \(\square\)

Property 2 (Preserves norm): \(\|U\mathbf{x}\| = \|\mathbf{x}\|\)

Proof: \(\|U\mathbf{x}\|^2 = \langle U\mathbf{x}, U\mathbf{x} \rangle = \langle \mathbf{x}, \mathbf{x} \rangle = \|\mathbf{x}\|^2\). \(\square\)

Geometrically, unitary transformations are isometries — they rotate and reflect complex space without any stretching.

Property 3 (Eigenvalues on the unit circle): Every eigenvalue \(\lambda\) satisfies \(|\lambda| = 1\).

Proof: If \(U\mathbf{v} = \lambda\mathbf{v}\), then \(\|U\mathbf{v}\| = |\lambda|\|\mathbf{v}\|\). But Property 2 gives \(\|U\mathbf{v}\| = \|\mathbf{v}\|\). Since \(\|\mathbf{v}\| \neq 0\): \(|\lambda| = 1\). \(\square\)

Thus \(\lambda = e^{i\theta}\) for some \(\theta \in \mathbb{R}\).

Property 4 (Determinant has unit modulus): \(|\det(U)| = 1\).

Proof: \(|\det(U)|^2 = \det(U)\cdot\overline{\det(U)} = \det(U)\cdot\det(U^H) = \det(UU^H) = \det(I) = 1\). \(\square\)

1.7 The Four Fundamental Subspaces in the Complex Case

For \(A \in \mathbb{C}^{m \times n}\), the four fundamental subspaces use \(A^H\) wherever the real theory used \(A^T\):

• Column Space \(\mathcal{R}(A) \subset \mathbb{C}^m\)
• Left Nullspace \(\mathcal{N}(A^H) \subset \mathbb{C}^m\)
• Row Space \(\mathcal{R}(A^H) \subset \mathbb{C}^n\)
• Nullspace \(\mathcal{N}(A) \subset \mathbb{C}^n\)

The orthogonality relationships become: \[\mathcal{N}(A) = \mathcal{R}(A^H)^\perp \quad \text{and} \quad \mathcal{N}(A^H) = \mathcal{R}(A)^\perp\]

where orthogonality uses the Hermitian inner product \(\langle \mathbf{x}, \mathbf{y} \rangle = \mathbf{x}^H\mathbf{y}\).

Critical difference from the real case: The null space \(\mathcal{N}(A)\) is orthogonal to \(\mathcal{R}(A^H)\), not to \(\mathcal{R}(A^T)\). Using the ordinary transpose instead of the conjugate transpose gives the wrong orthogonality.

1.8 Eigenvalues and Diagonalization over \(\mathbb{C}\)

Eigenvalues always exist over \(\mathbb{C}\): By the Fundamental Theorem of Algebra, the characteristic polynomial \(\det(A - \lambda I)\) of degree \(n\) has exactly \(n\) complex roots. So every \(n \times n\) matrix over \(\mathbb{C}\) has at least one eigenvalue.

Diagonalizability criterion (same as over \(\mathbb{R}\)): \(A\) is diagonalizable if and only if the algebraic multiplicity equals the geometric multiplicity for every eigenvalue. Equivalently, \(A\) has \(n\) linearly independent eigenvectors.

Normal Matrices: A matrix \(A \in \mathbb{C}^{n \times n}\) is called normal if: \[AA^H = A^H A\]

Normal matrices are precisely the matrices that are unitarily diagonalizable: there exists a unitary \(U\) and diagonal \(\Lambda\) such that \(A = U\Lambda U^H\).

Both Hermitian and unitary matrices are normal. The unitary diagonalization \(A = U\Lambda U^H\) is the complex analog of the orthogonal diagonalization \(A = Q\Lambda Q^T\) for real symmetric matrices.

1.9 Applications

Complex matrices and the spectral theory developed in this article appear throughout modern science and engineering.

Quantum Mechanics:

• Physical states are unit vectors in a complex Hilbert space (\(\mathbb{C}^n\) for finite-dimensional systems).
• Observables (measurable physical quantities such as energy or spin) are represented by Hermitian operators. Their real eigenvalues are the only values that can be observed in an experiment — the reality of eigenvalues is not merely a mathematical nicety but a physical necessity.
• Time evolution is governed by the unitary operator \(U(t) = e^{-iHt/\hbar}\), where \(H\) is the Hermitian Hamiltonian and \(\hbar\) is the reduced Planck constant. Unitarity ensures that the total probability (the squared norm of the state vector) is conserved at all times.

Signal Processing:

• The \(n\times n\) Discrete Fourier Transform (DFT) matrix \(F_n\) with entries \((F_n)_{jk} = e^{-2\pi ijk/n}\) is unitary up to scaling (\(F_n^H F_n = nI\)). Its columns form an orthogonal basis of complex exponentials.
• The Fast Fourier Transform (FFT) exploits the recursive structure of \(F_n\) to reduce the number of operations from \(O(n^2)\) to \(O(n\log n)\), enabling real-time digital signal processing.

Control Theory:

• The stability of a continuous-time linear system \(\dot{\mathbf{x}} = A\mathbf{x}\) is determined entirely by the eigenvalues of \(A\): the system is asymptotically stable if and only if all eigenvalues have strictly negative real parts. Complex eigenvalues \(\lambda = \sigma \pm i\omega\) correspond to oscillatory behavior with damping \(e^{\sigma t}\) and frequency \(\omega\).

Concept	Real Field (\(\mathbb{R}\))	Complex Field (\(\mathbb{C}\))
Vector space	\(\mathbb{R}^n\)	\(\mathbb{C}^n\)
Inner product	\(\mathbf{x}^T\mathbf{y}\)	\(\mathbf{x}^H\mathbf{y} = \sum x_i\overline{y_i}\)
Norm	\(\sqrt{\sum x_i^2}\)	\(\sqrt{\sum |x_i|^2}\)
Transpose analog	\(A^T\)	\(A^H\) (conjugate transpose)
Symmetric matrix	\(A^T = A\)	Hermitian: \(A^H = A\)
Orthogonal matrix	\(Q^T = Q^{-1}\)	Unitary: \(U^H = U^{-1}\)
Spectral Theorem	\(A = Q\Lambda Q^T\)	\(A = U\Lambda U^H\)
Eigenvalues guaranteed	No (real field only)	Yes (always exist over \(\mathbb{C}\))

---

2. Definitions

• Similar Matrices: Matrices \(A\) and \(B\) are similar (\(A \sim B\)) if there exists an invertible matrix \(P\) such that \(B = P^{-1}AP\). Similar matrices represent the same linear transformation in different bases.
• Invariants of Similarity: Properties shared by all similar matrices: determinant, trace, characteristic polynomial, eigenvalues (with multiplicities).
• Complex Number: A number of the form \(z = a + bi\) where \(a, b \in \mathbb{R}\) and \(i^2 = -1\).
• Complex Conjugate: For \(z = a + bi\), the conjugate is \(\bar{z} = a - bi\). Key property: \(z\bar{z} = |z|^2 \geq 0\).
• Modulus of a Complex Number: \(|z| = \sqrt{a^2 + b^2}\), the distance from \(z\) to the origin in the complex plane.
• Polar Form: \(z = re^{i\theta}\) where \(r = |z|\) and \(\theta = \arg(z)\). Follows from Euler’s formula \(e^{i\theta} = \cos\theta + i\sin\theta\).
• Complex Vector: An element of \(\mathbb{C}^n\): an ordered \(n\)-tuple of complex numbers.
• Hermitian Inner Product: \(\langle \mathbf{x}, \mathbf{y} \rangle = \mathbf{x}^H\mathbf{y} = \sum_{i=1}^n x_i\overline{y_i}\). Satisfies conjugate symmetry, linearity in first argument, conjugate linearity in second, and positive definiteness.
• Conjugate Transpose (Hermitian Adjoint): For \(A \in \mathbb{C}^{m \times n}\), the matrix \(A^H \in \mathbb{C}^{n \times m}\) defined by \((A^H)_{ij} = \overline{A_{ji}}\). Also written \(A^*\).
• Hermitian Matrix: A square complex matrix satisfying \(A = A^H\). The complex analog of a real symmetric matrix. Has real eigenvalues and orthogonal eigenvectors for distinct eigenvalues.
• Unitary Matrix: A square complex matrix \(U\) satisfying \(U^H U = I\) (equivalently, \(U^{-1} = U^H\)). The complex analog of a real orthogonal matrix. Preserves inner products and norms.
• Normal Matrix: A matrix satisfying \(AA^H = A^H A\). Characterized by being unitarily diagonalizable. Both Hermitian and unitary matrices are normal.
• Sesquilinear Form: A map linear in one argument and conjugate-linear in the other. The Hermitian inner product is sesquilinear (as opposed to bilinear in the real case).
• Unitary Diagonalization: Writing \(A = U\Lambda U^H\) where \(U\) is unitary and \(\Lambda\) is diagonal. Possible if and only if \(A\) is normal.

---

3. Formulas

• Similar Matrices: \(B = P^{-1}AP\) (equivalently \(D = P^{-1}AP\) for diagonalization)
• Invariants of Similarity: \(\det(B) = \det(A)\), \(\text{tr}(B) = \text{tr}(A)\), same characteristic polynomial and eigenvalues
• Powers of Similar Matrices: \((P^{-1}AP)^n = P^{-1}A^nP\)
• Complex Conjugate Properties: \(z + \bar{z} = 2\,\text{Re}(z)\); \(z - \bar{z} = 2i\,\text{Im}(z)\); \(z\bar{z} = |z|^2\)
• Polar Form: \(z = re^{i\theta} = r(\cos\theta + i\sin\theta)\) with \(r = |z|\), \(\theta = \arg(z)\)
• Powers in Polar Form: \(z^n = r^n e^{in\theta}\)
• Hermitian Inner Product: \(\langle \mathbf{x}, \mathbf{y} \rangle = \mathbf{x}^H\mathbf{y} = \sum_{i=1}^n x_i\overline{y_i}\)
• Conjugate Symmetry: \(\langle \mathbf{x}, \mathbf{y} \rangle = \overline{\langle \mathbf{y}, \mathbf{x} \rangle}\)
• Norm in \(\mathbb{C}^n\): \(\|\mathbf{v}\| = \sqrt{\langle \mathbf{v},\mathbf{v} \rangle} = \sqrt{\sum_{i=1}^n |v_i|^2}\)
• Cauchy-Schwarz Inequality: \(|\langle \mathbf{x}, \mathbf{y} \rangle| \leq \|\mathbf{x}\|\|\mathbf{y}\|\)
• Conjugate Transpose: \((A^H)_{ij} = \overline{A_{ji}}\); properties: \((A^H)^H = A\), \((AB)^H = B^H A^H\), \((\alpha A)^H = \bar{\alpha}A^H\)
• Adjoint Identity: \(\langle A\mathbf{x}, \mathbf{y} \rangle = \langle \mathbf{x}, A^H\mathbf{y} \rangle\)
• Hermitian Condition: \(A = A^H\) (diagonal entries real, \(a_{ij} = \overline{a_{ji}}\))
• Unitary Condition: \(U^H U = U U^H = I\) (equivalently \(U^{-1} = U^H\))
• Unitary Diagonalization (Spectral Theorem): \(A = U\Lambda U^H\) (valid for Hermitian and all normal matrices)
• Normal Matrix Condition: \(AA^H = A^H A\)
• Eigenvalues of Hermitian Matrix: Always real (\(\lambda = \bar{\lambda}\))
• Eigenvalues of Unitary Matrix: All satisfy \(|\lambda| = 1\) (lie on the unit circle)
• Determinant of Unitary Matrix: \(|\det(U)| = 1\)
• Orthogonality in Complex Case: \(\mathcal{N}(A) \perp \mathcal{R}(A^H)\) (not \(\mathcal{R}(A^T)\))

---

4. Examples

4.1. Basic Complex Arithmetic (Lab 7, Task 1)

Add and multiply each pair of complex numbers: (a) \(2+i\) and \(2-i\) (b) \(-1+i\) and \(-1+i\) (c) \(\cos\theta + i\sin\theta\) and \(\cos\theta - i\sin\theta\)

Click to see the solution

Key Concept: Use standard complex arithmetic: \((a+bi)+(c+di)=(a+c)+(b+d)i\) and \((a+bi)(c+di)=(ac-bd)+(ad+bc)i\).

(a) \((2+i)+(2-i)=4\); \((2+i)(2-i)=4+1=5\)

(b) \((-1+i)+(-1+i)=-2+2i\); \((-1+i)^2=1-2i-1=-2i\)

(c) \((\cos\theta+i\sin\theta)+(\cos\theta-i\sin\theta)=2\cos\theta\); \((\cos\theta+i\sin\theta)(\cos\theta-i\sin\theta)=\cos^2\theta+\sin^2\theta=1\)

Answer: (a) sum \(4\), product \(5\); (b) sum \(-2+2i\), product \(-2i\); (c) sum \(2\cos\theta\), product \(1\).

4.2. Properties of the Complex Conjugate (Lab 7, Task 2)

Complete the statements. Given \(z = a + ib\): (a) \(z + \bar{z}\) is always … (b) \(z - \bar{z}\) is always … (c) \(z \cdot \bar{z}\) equals … (d) \(\dfrac{z}{\bar{z}}\) (when \(\bar{z} \neq 0\)) always has modulus …

Click to see the solution

Key Concept: \(\bar{z} = a - ib\), so arithmetic gives real or purely imaginary results.

(a) \(z + \bar{z} = 2a\) — always real.

(b) \(z - \bar{z} = 2bi\) — always purely imaginary (or zero).

(c) \(z\bar{z} = a^2 + b^2 = |z|^2\) — the squared modulus (real, non-negative).

(d) \(\left|\dfrac{z}{\bar{z}}\right| = \dfrac{|z|}{|\bar{z}|} = 1\) — modulus always equals 1.

Answer: (a) real; (b) purely imaginary; (c) \(|z|^2 = a^2+b^2\); (d) \(1\).

4.3. Norms and Inner Products of Complex Vectors (Lab 7, Task 3)

Find the lengths (norms) of \(u, v \in \mathbb{C}^3\) and compute \(u^H v\) and \(v^H u\): \[u = \begin{pmatrix} 1+i \\ 1-i \\ 1+2i \end{pmatrix}, \quad v = \begin{pmatrix} i \\ i \\ i \end{pmatrix}\]

Click to see the solution

Key Concept: For complex vectors, \(\|w\|^2 = w^H w = \sum |\bar{w}_k w_k|\). The Hermitian inner product \(u^H v = \sum \bar{u}_k v_k\).

1. Norm of \(u\): \(\|u\|^2 = |1+i|^2 + |1-i|^2 + |1+2i|^2 = 2 + 2 + 5 = 9\), so \(\|u\| = 3\).
2. Norm of \(v\): \(\|v\|^2 = |i|^2 \cdot 3 = 3\), so \(\|v\| = \sqrt{3}\).
3. \(u^H v\): \(\bar{u}^T v = (1-i)(i) + (1+i)(i) + (1-2i)(i) = (i+1) + (i-1) + (i+2) = 3i+2\).
4. \(v^H u\): \(\overline{u^H v} = 3(-i)+2 = 2-3i\).

Answer: \(\|u\|=3\), \(\|v\|=\sqrt{3}\), \(u^H v = 2+3i\), \(v^H u = 2-3i\).

4.4. Properties of a Complex Matrix (Lab 7, Task 4)

Classify the matrix \(P\) (Hermitian? Unitary? Invertible?): \[P = \begin{pmatrix} 0 & i & 0 \\ 0 & 0 & i \\ i & 0 & 0 \end{pmatrix}\] Compute \(P^2\), \(P^3\), and \(P^{100}\).

Click to see the solution

Key Concept: A matrix is unitary if \(P^H P = I\); Hermitian if \(P^H = P\).

1. \(P^H\): Conjugate transpose gives \(P^H = \begin{pmatrix}0&0&-i\\-i&0&0\\0&-i&0\end{pmatrix}\). Since \(P^H \neq P\), \(P\) is not Hermitian. Check \(P^H P\): each column of \(P\) has norm \(1\) and columns are orthogonal → \(P^H P = I\), so \(P\) is unitary (and hence invertible).

2. \(P^2 = P \cdot P\): \[P^2 = \begin{pmatrix}0&0&-1\\-1&0&0\\0&-1&0\end{pmatrix}\]

3. \(P^3 = P^2 \cdot P\): Write \(P = iQ\) where \(Q = \begin{pmatrix}0&1&0\\0&0&1\\1&0&0\end{pmatrix}\) is the cyclic permutation. Then \(Q^3 = I\), so: \[P^3 = (iQ)^3 = i^3 Q^3 = -i \cdot I = -iI\]

4. \(P^{100}\): The order: \(P^6 = (-iI)^2 = -I\), \(P^{12} = I\). Since \(100 = 12\cdot8 + 4\): \[P^{100} = P^4 = P^3 \cdot P = -iI \cdot P = -iP\]

Answer: \(P\) is unitary (not Hermitian), invertible. \(P^2 = \begin{pmatrix}0&0&-1\\-1&0&0\\0&-1&0\end{pmatrix}\), \(P^3 = -iI\), \(P^{100} = -iP\).

4.5. Diagonalization to Prove a Matrix Power Formula (Lab 7, Task 5)

Diagonalize \(B = \begin{pmatrix}3&1\\0&2\end{pmatrix}\) to prove: \[B^k = \begin{pmatrix}3^k & 3^k-2^k \\ 0 & 2^k\end{pmatrix}\]

Click to see the solution

Key Concept: Upper triangular matrix → eigenvalues are diagonal entries. Use \(B^k = S\Lambda^k S^{-1}\).

1. Eigenvalues: \(\lambda_1=3\), \(\lambda_2=2\).
2. Eigenvectors:
   • \(\lambda_1=3\): \((B-3I)\mathbf{v}=0 \Rightarrow \mathbf{v}_1=\begin{pmatrix}1\\0\end{pmatrix}\)
   • \(\lambda_2=2\): \((B-2I)\mathbf{v}=0 \Rightarrow \begin{pmatrix}1&1\\0&0\end{pmatrix}\mathbf{v}=0 \Rightarrow \mathbf{v}_2=\begin{pmatrix}1\\-1\end{pmatrix}\)
3. \(S=\begin{pmatrix}1&1\\0&-1\end{pmatrix}\), \(S^{-1}=\begin{pmatrix}1&1\\0&-1\end{pmatrix}\), \(\Lambda=\begin{pmatrix}3&0\\0&2\end{pmatrix}\).
4. \(B^k = S\Lambda^k S^{-1} = \begin{pmatrix}1&1\\0&-1\end{pmatrix}\begin{pmatrix}3^k&0\\0&2^k\end{pmatrix}\begin{pmatrix}1&1\\0&-1\end{pmatrix} = \begin{pmatrix}3^k&3^k-2^k\\0&2^k\end{pmatrix}\) ✓

Answer: \(B^k = \begin{pmatrix}3^k&3^k-2^k\\0&2^k\end{pmatrix}\).

4.6. Construct Matrix from Eigenvalues (Lab 7, Task 6)

Find the second row of \(A = \begin{pmatrix}1&1\\\square&\square\end{pmatrix}\) so that its eigenvalues are \(4\) and \(7\).

Click to see the solution

Key Concept: Use trace and determinant: \(\lambda_1+\lambda_2 = \text{tr}(A)\) and \(\lambda_1\lambda_2 = \det(A)\).

1. Trace: \(\text{tr}(A) = 1 + a_{22} = 4+7=11\), so \(a_{22}=10\).
2. Determinant: \(\det(A) = 1\cdot10 - 1\cdot a_{21} = 4\cdot7=28\), so \(a_{21} = 10-28=-18\).

Answer: \(A = \begin{pmatrix}1&1\\-18&10\end{pmatrix}\).

4.7. Reconstruct Matrix from Eigendata (Lab 7, Task 7)

Find the matrix \(A\) with eigenvalues \(\lambda_1=1\), \(\lambda_2=4\) and corresponding eigenvectors \((3,1)^T\) and \((2,1)^T\).

Click to see the solution

Key Concept: \(A = S\Lambda S^{-1}\) where columns of \(S\) are eigenvectors and \(\Lambda\) is diagonal with eigenvalues.

1. \(S = \begin{bmatrix}3&2\\1&1\end{bmatrix}\), \(\Lambda = \begin{bmatrix}1&0\\0&4\end{bmatrix}\), \(S^{-1} = \begin{bmatrix}1&-2\\-1&3\end{bmatrix}\).
2. \(A = S\Lambda S^{-1} = \begin{bmatrix}3&2\\1&1\end{bmatrix}\begin{bmatrix}1&0\\0&4\end{bmatrix}\begin{bmatrix}1&-2\\-1&3\end{bmatrix} = \begin{bmatrix}3&8\\1&4\end{bmatrix}\begin{bmatrix}1&-2\\-1&3\end{bmatrix} = \begin{bmatrix}-5&18\\-3&10\end{bmatrix}\).

Answer: \(A = \begin{bmatrix}-5&18\\-3&10\end{bmatrix}\).

4.8. Polar Form of Complex Numbers (Lab 7, Task 8)

If \(x = re^{i\theta}\), find \(x^2\), \(x^{-1}\), and \(\bar{x}\) in polar form. On what geometric figure do complex numbers satisfying \(x^{-1} = \bar{x}\) lie?

Click to see the solution

Key Concept: In polar form \(re^{i\theta}\), powers multiply the modulus and add the argument.

1. \(x^2 = r^2 e^{i2\theta}\)
2. \(x^{-1} = \frac{1}{r}e^{-i\theta}\)
3. \(\bar{x} = re^{-i\theta}\)
4. Condition \(x^{-1} = \bar{x}\): \(\frac{1}{r}e^{-i\theta} = re^{-i\theta}\), so \(r = 1\). These are complex numbers with modulus 1 → they lie on the unit circle.

Answer: \(x^2=r^2e^{2i\theta}\), \(x^{-1}=\frac{1}{r}e^{-i\theta}\), \(\bar{x}=re^{-i\theta}\). The condition \(x^{-1}=\bar{x}\) defines the unit circle \(|x|=1\).

4.9. Hermitian Conjugate and \(A^H A\) (Lab 7, Task 9)

For \(A = \begin{bmatrix}1&i&0\\i&0&1\end{bmatrix}\), write \(A^H\) and compute \(C = A^H A\). What is the relationship between \(C\) and \(C^H\)?

Click to see the solution

Key Concept: \(A^H = \overline{A}^T\). The product \(A^H A\) is always Hermitian: \((A^H A)^H = A^H A\).

1. \(A^H = \begin{bmatrix}1&-i\\-i&0\\0&1\end{bmatrix}\)
2. \(C = A^H A = \begin{bmatrix}1&-i\\-i&0\\0&1\end{bmatrix}\begin{bmatrix}1&i&0\\i&0&1\end{bmatrix} = \begin{bmatrix}2&i&-i\\-i&1&0\\i&0&1\end{bmatrix}\)
3. \(C^H = \overline{C}^T = C\) → \(C\) is Hermitian.

Answer: \(A^H = \begin{bmatrix}1&-i\\-i&0\\0&1\end{bmatrix}\), \(C = \begin{bmatrix}2&i&-i\\-i&1&0\\i&0&1\end{bmatrix}\), and \(C = C^H\) (Hermitian).

4.10. Null Space and Complex Orthogonality (Lab 7, Task 10)

Using \(A = \begin{bmatrix}1&i&0\\i&0&1\end{bmatrix}\): (a) Solve \(A\mathbf{x} = \mathbf{0}\) using Gaussian elimination. (b) Show that \(N(A)\) is orthogonal to \(C(A^H)\), not to \(C(A^T)\).

Click to see the solution

Key Concept: In the complex case, the four fundamental subspaces use \(A^H\) instead of \(A^T\).

(a) Row reduce \(A\): \[\begin{bmatrix}1&i&0\\i&0&1\end{bmatrix} \xrightarrow{R_2 - iR_1} \begin{bmatrix}1&i&0\\0&1&1\end{bmatrix} \xrightarrow{R_1 - iR_2} \begin{bmatrix}1&0&-i\\0&1&1\end{bmatrix}\] Free variable \(x_3 = t\): \(x_1 = it\), \(x_2 = -t\). Null space: \(\mathbf{n} = t\begin{pmatrix}i\\-1\\1\end{pmatrix}\).

(b) Columns of \(A^H\): \(\mathbf{c}_1=\begin{pmatrix}1\\-i\\0\end{pmatrix}\), \(\mathbf{c}_2=\begin{pmatrix}-i\\0\\1\end{pmatrix}\). \(\mathbf{n}^H \mathbf{c}_1 = \bar{t}(-i-i\cdot(-i\cdot i)+0) = \bar{t}(1i^* + i\cdot i \ldots)\) — compute directly: \(\overline{(i,-1,1)}\cdot(1,-i,0) = (-i)(-i,)+(−1)(−i)+(1)(0)= -1+i+0\)…

Direct check: \(A\mathbf{n} = \mathbf{0}\) means \(\mathbf{n} \in N(A)\). By the fundamental theorem, \(N(A) \perp C(A^H)\) since \((A^H \mathbf{y})^H \mathbf{n} = \mathbf{y}^H(A\mathbf{n}) = 0\).

Answer: \(N(A) = \text{span}\left\{\begin{pmatrix}i\\-1\\1\end{pmatrix}\right\}\). Orthogonality \(N(A) \perp C(A^H)\) follows from \(A\mathbf{n}=\mathbf{0}\).

4.11. Matrix Similarity (Lab 7, Task 11)

Show that the following pairs of matrices are similar by finding \(M\) such that \(B = M^{-1}AM\):

(a) \(A = \begin{bmatrix}1&0\\1&0\end{bmatrix}\), \(B = \begin{bmatrix}1&0\\0&0\end{bmatrix}\)

(b) \(A = \begin{bmatrix}1&1\\1&1\end{bmatrix}\), \(B = \begin{bmatrix}1&-1\\-1&1\end{bmatrix}\)

(c) \(A = \begin{bmatrix}1&2\\3&4\end{bmatrix}\), \(B = \begin{bmatrix}4&3\\2&1\end{bmatrix}\)

Click to see the solution

Key Concept: \(B = M^{-1}AM \Leftrightarrow MA = MB\), so we need to find invertible \(M\). Similar matrices share eigenvalues.

(a) Both have eigenvalues \(0\) and \(1\). Try \(M = \begin{bmatrix}1&0\\0&1\end{bmatrix}\)… actually \(M = \begin{bmatrix}0&1\\1&0\end{bmatrix}\) (swap columns): verify \(M^{-1}AM = B\). \(M^{-1} = M\), \(MAM = \begin{bmatrix}0&1\\1&0\end{bmatrix}\begin{bmatrix}1&0\\1&0\end{bmatrix}\begin{bmatrix}0&1\\1&0\end{bmatrix} = \begin{bmatrix}1&0\\1&0\end{bmatrix}\begin{bmatrix}0&1\\1&0\end{bmatrix}=\begin{bmatrix}0&1\\0&1\end{bmatrix}\). Try \(M=\begin{bmatrix}1&1\\0&1\end{bmatrix}\): \(M^{-1}AM = \begin{bmatrix}1&0\\0&0\end{bmatrix}\) ✓

(b) Both have eigenvalues \(0\) and \(2\). \(M = \begin{bmatrix}1&0\\0&-1\end{bmatrix}\) works: \(M^{-1}=M\), \(MAM = M\begin{bmatrix}1&1\\1&1\end{bmatrix}M = \begin{bmatrix}1&-1\\-1&1\end{bmatrix}\) ✓

(c) \(A\) and \(B = A^T\) — any matrix with the same eigenvalues and \(B=PAP^{-1}\) for a permutation-like \(M\). Since \(\text{tr}(A)=\text{tr}(B)=5\) and \(\det(A)=\det(B)=-2\), they share eigenvalues. One valid \(M\): \(M = \begin{bmatrix}0&1\\1&0\end{bmatrix}\) (swap rows/columns gives transpose): verify \(M^{-1}AM = A^T = B\) ✓

Answer: (a) \(M=\begin{bmatrix}1&1\\0&1\end{bmatrix}\); (b) \(M=\begin{bmatrix}1&0\\0&-1\end{bmatrix}\); (c) \(M=\begin{bmatrix}0&1\\1&0\end{bmatrix}\).

4.12. Determine Whether Pairs of Matrices Are Similar (Assignment 7, Task 1.1)

Determine whether the following pairs of matrices are similar. Justify your answer.

(a) \(A = \begin{pmatrix} 2 & 1 \\ 0 & 2 \end{pmatrix}\), \(B = \begin{pmatrix} 2 & 0 \\ 1 & 2 \end{pmatrix}\)

(b) \(A = \begin{pmatrix} 1 & 2 \\ 0 & 3 \end{pmatrix}\), \(B = \begin{pmatrix} 3 & 0 \\ 0 & 1 \end{pmatrix}\)

(c) \(A = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}\), \(B = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}\)

Click to see the solution

Key Concept: To show similarity, find an invertible \(M\) with \(MA = BM\). To show non-similarity, show that no invertible \(M\) can exist.

(a) \(A \sim B\): Setting \(MA = BM\) with \(M = \begin{pmatrix} a & b \\ c & d \end{pmatrix}\) gives \(a = 0\) and \(c = b\). Choose \(b = 1\), \(d = 0\): \(M = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}\), \(\det(M) = -1 \neq 0\).

Verify: \(MA = \begin{pmatrix} 0 & 2 \\ 2 & 1 \end{pmatrix}\), \(BM = \begin{pmatrix} 0 & 2 \\ 2 & 1 \end{pmatrix}\) ✓.

(b) \(A \sim B\): Both \(A\) and \(B\) have eigenvalues \(\lambda = 1, 3\) (distinct). A matrix with distinct eigenvalues is diagonalizable, and any two diagonalizable matrices with the same eigenvalues are similar to the same diagonal matrix, hence similar to each other. Choose \(M = \begin{pmatrix} 0 & 1 \\ 1 & -1 \end{pmatrix}\) (\(\det(M) = -1\)).

Verify: \(MA = BM\) ✓.

(c) \(A \not\sim B\): \(B = I\) is only similar to itself — for any invertible \(P\), \(P^{-1}IP = I \neq A\). To see why directly: \(MA = BM = M\) requires \(M(A - I) = 0\). Since \(A - I = \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix} \neq 0\), any \(M\) satisfying this is singular. No invertible \(M\) exists.

Answer: (a) Similar, \(M = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}\); (b) Similar, \(M = \begin{pmatrix} 0 & 1 \\ 1 & -1 \end{pmatrix}\); (c) Not similar.

4.13. Prove Invariants of Similar Matrices (Assignment 7, Task 1.2)

Let \(B = P^{-1}AP\) for some invertible matrix \(P\). Prove:

(a) \(\det(A) = \det(B)\)

(b) \(\text{tr}(A) = \text{tr}(B)\)

(c) \(A\) and \(B\) have the same characteristic polynomial

(d) \(A\) and \(B\) have the same eigenvalues with the same multiplicities

Click to see the solution

(a) \(\det(B) = \det(P^{-1})\det(A)\det(P) = \dfrac{\det(A)}{\det(P)}\cdot\det(P) = \det(A)\). \(\square\)

(b) \(\text{tr}(P^{-1}AP) = \text{tr}(APP^{-1}) = \text{tr}(A)\) (cyclic property of trace). \(\square\)

(c) \(\det(B - \lambda I) = \det(P^{-1}(A-\lambda I)P) = \det(P^{-1})\det(A-\lambda I)\det(P) = \det(A - \lambda I)\). \(\square\)

(d) The eigenvalues are the roots of the characteristic polynomial. Since (c) shows the polynomials are equal, the roots — and their multiplicities — are the same. \(\square\)

Answer: All four follow from multiplicativity of \(\det\) and cyclicity of \(\text{tr}\).

4.14. Diagonalize a \(3 \times 3\) Matrix (Assignment 7, Task 1.3)

Find a matrix \(P\) that diagonalizes \(A\) and verify \(P^{-1}AP\) is diagonal: \[A = \begin{pmatrix} 2 & 0 & 0 \\ 1 & 2 & 1 \\ -1 & 0 & 1 \end{pmatrix}\]

Is this diagonalization unique?

Click to see the solution

Step 1: Find eigenvalues.

\[\det(A - \lambda I) = (2-\lambda)\det\begin{pmatrix} 2-\lambda & 1 \\ 0 & 1-\lambda \end{pmatrix} = (2-\lambda)^2(1-\lambda) = 0\]

Eigenvalues: \(\lambda = 2\) (mult. 2) and \(\lambda = 1\) (mult. 1).

Step 2: Eigenvectors for \(\lambda = 2\).

\[A - 2I = \begin{pmatrix} 0 & 0 & 0 \\ 1 & 0 & 1 \\ -1 & 0 & -1 \end{pmatrix} \implies x_1 + x_3 = 0,\; x_2\text{ free}\]

Basis: \(\mathbf{v}_1 = \begin{pmatrix} 0 \\ 1 \\ 0 \end{pmatrix}\), \(\mathbf{v}_2 = \begin{pmatrix} -1 \\ 0 \\ 1 \end{pmatrix}\). Geometric multiplicity = 2 = algebraic multiplicity ✓.

Step 3: Eigenvector for \(\lambda = 1\).

\[A - I = \begin{pmatrix} 1 & 0 & 0 \\ 1 & 1 & 1 \\ -1 & 0 & 0 \end{pmatrix} \implies x_1 = 0,\; x_2 = -x_3\]

Eigenvector: \(\mathbf{v}_3 = \begin{pmatrix} 0 \\ -1 \\ 1 \end{pmatrix}\).

Result:

\[P = \begin{pmatrix} 0 & -1 & 0 \\ 1 & 0 & -1 \\ 0 & 1 & 1 \end{pmatrix}, \quad D = \begin{pmatrix} 2 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 1 \end{pmatrix}\]

Not unique: Any nonzero linear combination of \(\mathbf{v}_1, \mathbf{v}_2\) is a valid eigenvector for \(\lambda = 2\), giving infinitely many valid \(P\).

Answer: \(P = \begin{pmatrix} 0 & -1 & 0 \\ 1 & 0 & -1 \\ 0 & 1 & 1 \end{pmatrix}\), \(D = \text{diag}(2, 2, 1)\). Not unique.

4.15. Compute \(A^{10}\) Using Diagonalization (Assignment 7, Task 1.4)

Show that if \(A \sim B\) then \(A^n \sim B^n\). Use this to compute \(A^{10}\) for \(A = \begin{pmatrix} 4 & 3 \\ -2 & -1 \end{pmatrix}\).

Click to see the solution

Proof: If \(B = P^{-1}AP\), then \(B^n = (P^{-1}AP)^n = P^{-1}A^nP\) (since \(P^{-1}P = I\) telescopes). \(\square\)

Diagonalize \(A\): \(\det(A-\lambda I) = (4-\lambda)(-1-\lambda)+6 = \lambda^2-3\lambda+2 = (\lambda-1)(\lambda-2) = 0\).

For \(\lambda_1 = 1\): \(\mathbf{v}_1 = \begin{pmatrix} 1 \\ -1 \end{pmatrix}\). For \(\lambda_2 = 2\): \(\mathbf{v}_2 = \begin{pmatrix} 3 \\ -2 \end{pmatrix}\).

\[P = \begin{pmatrix} 1 & 3 \\ -1 & -2 \end{pmatrix}, \quad D = \begin{pmatrix} 1 & 0 \\ 0 & 2 \end{pmatrix}, \quad P^{-1} = \begin{pmatrix} -2 & -3 \\ 1 & 1 \end{pmatrix}\]

\[A^{10} = PD^{10}P^{-1} = \begin{pmatrix} 1 & 3 \\ -1 & -2 \end{pmatrix}\begin{pmatrix} 1 & 0 \\ 0 & 1024 \end{pmatrix}\begin{pmatrix} -2 & -3 \\ 1 & 1 \end{pmatrix}\]

\[= \begin{pmatrix} 1 & 3072 \\ -1 & -2048 \end{pmatrix}\begin{pmatrix} -2 & -3 \\ 1 & 1 \end{pmatrix} = \begin{pmatrix} 3070 & 3069 \\ -2046 & -2045 \end{pmatrix}\]

Answer: \(A^{10} = \begin{pmatrix} 3070 & 3069 \\ -2046 & -2045 \end{pmatrix}\).

4.16. Find All Eigenvalues and Eigenvectors (Assignment 7, Task 2.1)

Find all eigenvalues and eigenvectors for:

(a) \(A = \begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix}\), (b) \(B = \begin{pmatrix} 2 & -1 \\ 1 & 2 \end{pmatrix}\), (c) \(C = \begin{pmatrix} 1 & 2 & 0 \\ 0 & 2 & 0 \\ 0 & 1 & 3 \end{pmatrix}\)

Click to see the solution

(a) \(\det(A-\lambda I) = (3-\lambda)^2 - 1 = (\lambda-4)(\lambda-2) = 0 \Rightarrow \lambda = 4, 2\).

\(\lambda=4\): \(v_1 = v_2\), eigenvector \(\begin{pmatrix}1\\1\end{pmatrix}\). \(\lambda=2\): \(v_1 = -v_2\), eigenvector \(\begin{pmatrix}1\\-1\end{pmatrix}\).

(b) \(\det(B-\lambda I) = (2-\lambda)^2 + 1 = 0 \Rightarrow \lambda = 2 \pm i\).

\(\lambda=2+i\): \(-iv_1 = v_2\), eigenvector \(\begin{pmatrix}1\\-i\end{pmatrix}\). \(\lambda=2-i\): eigenvector \(\begin{pmatrix}1\\i\end{pmatrix}\).

(c) Triangular matrix: \(\lambda_1=1\), \(\lambda_2=2\), \(\lambda_3=3\).

\(\lambda=1\): eigenvector \(\begin{pmatrix}1\\0\\0\end{pmatrix}\). \(\lambda=2\): row-reduce \((C-2I)\) to get \(\begin{pmatrix}2\\1\\-1\end{pmatrix}\). \(\lambda=3\): eigenvector \(\begin{pmatrix}0\\0\\1\end{pmatrix}\).

Answer: (a) \(\lambda=4\): \((1,1)^T\); \(\lambda=2\): \((1,-1)^T\). (b) \(\lambda=2\pm i\): \((1,\mp i)^T\). (c) \(\lambda=1,2,3\): as above.

4.17. True or False: Properties of Eigenvalues (Assignment 7, Task 2.2)

True or false (prove or give counterexample):

(a) Every square matrix has at least one real eigenvalue.

(b) The sum of two eigenvectors is always an eigenvector.

(c) If \(\lambda\) is an eigenvalue of \(A\), then \(\lambda^2\) is an eigenvalue of \(A^2\).

(d) \(A\) and \(A^T\) always have the same eigenvalues.

(e) If \(A\) is invertible, the eigenvalues of \(A^{-1}\) are the reciprocals of those of \(A\).

Click to see the solution

(a) FALSE. Counterexample: \(R = \begin{pmatrix}0&-1\\1&0\end{pmatrix}\) has characteristic polynomial \(\lambda^2+1=0\), no real roots.

(b) FALSE. If \(A\mathbf{v}_1=\lambda_1\mathbf{v}_1\) and \(A\mathbf{v}_2=\lambda_2\mathbf{v}_2\) with \(\lambda_1\neq\lambda_2\), then \(A(\mathbf{v}_1+\mathbf{v}_2)=\lambda_1\mathbf{v}_1+\lambda_2\mathbf{v}_2\), which is not a scalar multiple of \(\mathbf{v}_1+\mathbf{v}_2\).

(c) TRUE. \(A^2\mathbf{v}=A(\lambda\mathbf{v})=\lambda^2\mathbf{v}\). \(\square\)

(d) TRUE. \(\det(A^T-\lambda I)=\det((A-\lambda I)^T)=\det(A-\lambda I)\), same characteristic polynomial. \(\square\)

(e) TRUE. From \(A\mathbf{v}=\lambda\mathbf{v}\): \(\mathbf{v}=\lambda A^{-1}\mathbf{v}\), so \(A^{-1}\mathbf{v}=\frac{1}{\lambda}\mathbf{v}\) (with \(\lambda\neq0\) since \(A\) invertible). \(\square\)

Answer: (a) F, (b) F, (c) T, (d) T, (e) T.

4.18. Eigenvectors of Matrix Powers and Polynomials (Assignment 7, Task 2.3)

Let \(A\mathbf{v} = \lambda\mathbf{v}\) with \(\mathbf{v} \neq \mathbf{0}\). Show:

(a) \(\mathbf{v}\) is an eigenvector of \(A^2\) with eigenvalue \(\lambda^2\).

(b) \(\mathbf{v}\) is an eigenvector of \(A^k\) with eigenvalue \(\lambda^k\).

(c) \(\mathbf{v}\) is an eigenvector of \(p(A)\) with eigenvalue \(p(\lambda)\) for any polynomial \(p\).

Click to see the solution

(a) \(A^2\mathbf{v} = A(A\mathbf{v}) = A(\lambda\mathbf{v}) = \lambda(A\mathbf{v}) = \lambda^2\mathbf{v}\). \(\square\)

(b) By induction: Assume \(A^k\mathbf{v}=\lambda^k\mathbf{v}\). Then \(A^{k+1}\mathbf{v}=A(A^k\mathbf{v})=\lambda^k A\mathbf{v}=\lambda^{k+1}\mathbf{v}\). \(\square\)

(c) Let \(p(t)=\sum_{j=0}^m a_j t^j\). Then: \[p(A)\mathbf{v} = \sum_{j=0}^m a_j A^j\mathbf{v} = \sum_{j=0}^m a_j\lambda^j\mathbf{v} = p(\lambda)\mathbf{v} \quad \square\]

This is the Spectral Mapping Theorem: eigenvalues of \(p(A)\) are \(p(\lambda)\).

4.19. Using the Eigendecomposition to Compute \(A\mathbf{u}\) (Assignment 7, Task 2.4)

\(A\) is \(3\times3\) with eigenvalues \(\lambda_1=1\), \(\lambda_2=2\), \(\lambda_3=3\) and eigenvectors \(\mathbf{v}_1\), \(\mathbf{v}_2\), \(\mathbf{v}_3\).

(a) Find \(A\mathbf{v}_1\), \(A\mathbf{v}_2\), \(A\mathbf{v}_3\). (b) Find \(A^2\mathbf{v}_1\). (c) If \(\mathbf{u}=2\mathbf{v}_1-\mathbf{v}_2+3\mathbf{v}_3\), find \(A\mathbf{u}\). (d) What basis diagonalizes \(A\)?

Click to see the solution

(a) By definition: \(A\mathbf{v}_1=\mathbf{v}_1\), \(A\mathbf{v}_2=2\mathbf{v}_2\), \(A\mathbf{v}_3=3\mathbf{v}_3\).

(b) \(A^2\mathbf{v}_1=1^2\mathbf{v}_1=\mathbf{v}_1\).

(c) \(A\mathbf{u}=2A\mathbf{v}_1-A\mathbf{v}_2+3A\mathbf{v}_3=2\mathbf{v}_1-2\mathbf{v}_2+9\mathbf{v}_3\).

(d) The basis \(\{\mathbf{v}_1,\mathbf{v}_2,\mathbf{v}_3\}\) of eigenvectors. The diagonal matrix is \(D=\text{diag}(1,2,3)\).

4.20. Operations on Complex Vectors (Assignment 7, Task 3.1)

Let \(\mathbf{u}=\begin{pmatrix}1+i\\2-i\\3i\end{pmatrix}\), \(\mathbf{v}=\begin{pmatrix}2\\1-i\\1+2i\end{pmatrix}\), \(\mathbf{w}=\begin{pmatrix}i\\1\\-i\end{pmatrix}\).

(a) Compute \(\mathbf{u}+\mathbf{v}\), \(\mathbf{u}-\mathbf{w}\), \(2\mathbf{u}-3\mathbf{v}\). (b) Compute \(\langle\mathbf{u},\mathbf{v}\rangle=\mathbf{u}^H\mathbf{v}\). (c) Compute \(\langle\mathbf{v},\mathbf{u}\rangle\) and compare. (d) Find \(\|\mathbf{u}\|\), \(\|\mathbf{v}\|\), \(\|\mathbf{w}\|\). (e) Verify Cauchy-Schwarz for \(\mathbf{u}\) and \(\mathbf{v}\).

Click to see the solution

(a)

\[\mathbf{u}+\mathbf{v}=\begin{pmatrix}3+i\\3-2i\\1+5i\end{pmatrix},\quad \mathbf{u}-\mathbf{w}=\begin{pmatrix}1\\1-i\\4i\end{pmatrix},\quad 2\mathbf{u}-3\mathbf{v}=\begin{pmatrix}-4+2i\\1+i\\-3\end{pmatrix}\]

(b) \(\langle\mathbf{u},\mathbf{v}\rangle=\mathbf{u}^H\mathbf{v}=(1-i)(2)+(2+i)(1-i)+(-3i)(1+2i)=(2-2i)+(3-i)+(6-3i)=11-6i\).

(c) \(\langle\mathbf{v},\mathbf{u}\rangle=\overline{\langle\mathbf{u},\mathbf{v}\rangle}=11+6i\). Conjugate symmetry confirmed.

(d) \(\|\mathbf{u}\|^2=2+5+9=16\Rightarrow\|\mathbf{u}\|=4\); \(\|\mathbf{v}\|^2=4+2+5=11\Rightarrow\|\mathbf{v}\|=\sqrt{11}\); \(\|\mathbf{w}\|^2=3\Rightarrow\|\mathbf{w}\|=\sqrt{3}\).

(e) \(|\langle\mathbf{u},\mathbf{v}\rangle|=\sqrt{157}\approx12.53\leq4\sqrt{11}\approx13.27=\|\mathbf{u}\|\|\mathbf{v}\|\) ✓.

4.21. Eigenvalues of Complex Matrices (Assignment 7, Task 3.2)

Find all eigenvalues and eigenvectors:

(a) \(A=\begin{pmatrix}1&i\\-i&1\end{pmatrix}\), (b) \(B=\begin{pmatrix}0&1&0\\0&0&1\\1&0&0\end{pmatrix}\) (find complex eigenvalues), (c) \(C=\begin{pmatrix}\cos\theta&-\sin\theta\\\sin\theta&\cos\theta\end{pmatrix}\).

Click to see the solution

(a) \(\det(A-\lambda I)=(1-\lambda)^2-1=\lambda(\lambda-2)=0\). Eigenvalues: \(\lambda=0,2\). (Note: \(A\) is Hermitian, both real ✓.)

\(\lambda=0\): \(v_1+iv_2=0\), eigenvector \(\begin{pmatrix}i\\-1\end{pmatrix}\). \(\lambda=2\): \(-v_1+iv_2=0\), eigenvector \(\begin{pmatrix}i\\1\end{pmatrix}\).

(b) \(\det(B-\lambda I)=-\lambda^3+1=0\Rightarrow\lambda^3=1\). Cube roots of unity: \(\lambda_k=e^{2\pi ik/3}\), \(k=0,1,2\). Eigenvectors: \(\begin{pmatrix}1\\\lambda_k\\\lambda_k^2\end{pmatrix}\).

(c) \(\det(C-\lambda I)=\lambda^2-2\cos\theta\,\lambda+1=0\Rightarrow\lambda=e^{\pm i\theta}\). Eigenvectors: \(\begin{pmatrix}1\\\mp i\end{pmatrix}\).

4.22. Diagonalizability Over \(\mathbb{R}\) vs. \(\mathbb{C}\) (Assignment 7, Task 3.3)

Let \(A=\begin{pmatrix}1&1\\0&1\end{pmatrix}\) and \(B=\begin{pmatrix}1&i\\0&1\end{pmatrix}\).

(a) Find the eigenvalues of \(A\) and \(B\). (b) Find all eigenvectors. (c) Is either matrix diagonalizable? (d) What does this say about diagonalizability and the underlying field?

Click to see the solution

(a) Both triangular with diagonal entries 1, 1: eigenvalue \(\lambda=1\) with algebraic multiplicity 2 for both.

(b) For \(A\): \((A-I)=\begin{pmatrix}0&1\\0&0\end{pmatrix}\), so \(v_2=0\) and only \(\begin{pmatrix}t\\0\end{pmatrix}\) — one independent eigenvector. For \(B\): \((B-I)=\begin{pmatrix}0&i\\0&0\end{pmatrix}\), so \(iv_2=0\Rightarrow v_2=0\) — also only \(\begin{pmatrix}t\\0\end{pmatrix}\).

(c) Both need 2 linearly independent eigenvectors to be \(2\times2\) diagonalizable, but each has only 1 (geometric multiplicity 1 < algebraic multiplicity 2). Neither is diagonalizable — over \(\mathbb{R}\) or \(\mathbb{C}\).

(d) Working over \(\mathbb{C}\) guarantees eigenvalues exist, but does not fix defective matrices. Diagonalizability depends on whether geo. mult. = alg. mult. for every eigenvalue, regardless of field.

4.23. Unitary Matrices: Columns, Norm Preservation, Eigenvalues (Assignment 7, Task 3.4)

A matrix \(U\) is unitary if \(U^H U = I\).

(a) Show that the columns of a unitary matrix form an orthonormal basis in \(\mathbb{C}^n\).

(b) Show that \(\|U\mathbf{x}\| = \|\mathbf{x}\|\) for all \(\mathbf{x}\).

(c) Show that all eigenvalues of a unitary matrix satisfy \(|\lambda| = 1\).

(d) Verify that \(U = \dfrac{1}{\sqrt{2}}\begin{pmatrix}1&i\\i&1\end{pmatrix}\) is unitary and find its eigenvalues.

Click to see the solution

(a) Let \(\mathbf{u}_1, \ldots, \mathbf{u}_n\) be the columns of \(U\). Then \((U^H U)_{ij} = \mathbf{u}_i^H \mathbf{u}_j\). Since \(U^H U = I\), we have \(\mathbf{u}_i^H\mathbf{u}_j = \delta_{ij}\): orthonormal. \(\square\)

(b) \(\|U\mathbf{x}\|^2 = (U\mathbf{x})^H(U\mathbf{x}) = \mathbf{x}^H U^H U \mathbf{x} = \mathbf{x}^H\mathbf{x} = \|\mathbf{x}\|^2\). \(\square\)

(c) If \(U\mathbf{v}=\lambda\mathbf{v}\) with \(\mathbf{v}\neq\mathbf{0}\): \(\|\mathbf{v}\| = \|U\mathbf{v}\| = |\lambda|\|\mathbf{v}\| \Rightarrow |\lambda|=1\). \(\square\)

(d) Compute \(U^H = \frac{1}{\sqrt{2}}\begin{pmatrix}1&-i\\-i&1\end{pmatrix}\). Then \(U^HU = \frac{1}{2}\begin{pmatrix}2&0\\0&2\end{pmatrix}=I\) ✓ — unitary.

Eigenvalues: \(\lambda^2 - \sqrt{2}\,\lambda + 1 = 0 \Rightarrow \lambda = \dfrac{1\pm i}{\sqrt{2}}\), both with \(|\lambda|=1\) ✓.

Answer: (a)-(c) proved above. (d) \(U\) is unitary; eigenvalues \(\dfrac{1\pm i}{\sqrt{2}}\).

4.24. Conjugate Transpose and Hermitian Matrices (Assignment 7, Task 3.5)

For \(A = \begin{pmatrix}1&2\\3&4\end{pmatrix}\):

(a) Find \(A^H\) and \(A^T\). Are they the same? Why?

(b) Show \((A^H)^H = A\).

(c) Show \((AB)^H = B^H A^H\) for any compatible \(A\), \(B\).

(d) Give an example of a \(2\times2\) Hermitian matrix with complex entries.

Click to see the solution

(a) Since \(A\) has all real entries, conjugation changes nothing: \(A^H = A^T = \begin{pmatrix}1&3\\2&4\end{pmatrix}\). They are the same because \(A\) is real — for real matrices \(A^H = A^T\) always.

(b) \((A^H)^H\): taking conjugate transpose twice returns to the original. In components: \(((A^H)^H)_{ij} = \overline{(A^H)_{ji}} = \overline{\overline{A_{ij}}} = A_{ij}\). So \((A^H)^H = A\). \(\square\)

(c) \(((AB)^H)_{ij} = \overline{(AB)_{ji}} = \overline{\sum_k A_{jk}B_{ki}} = \sum_k \overline{B_{ki}}\,\overline{A_{jk}} = \sum_k (B^H)_{ik}(A^H)_{kj} = (B^H A^H)_{ij}\). \(\square\)

(d) Any matrix of the form \(\begin{pmatrix}a & b+ci \\ b-ci & d\end{pmatrix}\) with \(a,b,c,d\in\mathbb{R}\) is Hermitian. For example: \(\begin{pmatrix}2&1+i\\1-i&3\end{pmatrix}\). Check: diagonal entries real ✓; \((1,2)\)-entry \(= \overline{(2,1)\text{-entry}}\) ✓.

4.25. Determine Diagonalizability (Assignment 7, Task 4.1)

Determine whether each matrix is diagonalizable. If so, find \(P\) and \(D\) with \(P^{-1}AP = D\).

(a) \(A=\begin{pmatrix}3&1\\0&2\end{pmatrix}\), (b) \(B=\begin{pmatrix}2&1\\0&2\end{pmatrix}\), (c) \(C=\begin{pmatrix}1&0&0\\1&2&0\\1&0&3\end{pmatrix}\), (d) \(D=\begin{pmatrix}2&1&0\\0&2&1\\0&0&2\end{pmatrix}\)

Click to see the solution

(a) Diagonalizable. Eigenvalues 3 and 2 are distinct — automatically gives 2 independent eigenvectors.

\(\lambda=3\): \(\begin{pmatrix}0&1\\0&-1\end{pmatrix}\mathbf{v}=\mathbf{0}\Rightarrow\mathbf{v}_1=\begin{pmatrix}1\\0\end{pmatrix}\). \(\lambda=2\): \(\begin{pmatrix}1&1\\0&0\end{pmatrix}\mathbf{v}=\mathbf{0}\Rightarrow\mathbf{v}_2=\begin{pmatrix}-1\\1\end{pmatrix}\).

\(P=\begin{pmatrix}1&-1\\0&1\end{pmatrix}\), \(D=\begin{pmatrix}3&0\\0&2\end{pmatrix}\).

(b) Not diagonalizable. Eigenvalue 2 with algebraic multiplicity 2. \((B-2I)=\begin{pmatrix}0&1\\0&0\end{pmatrix}\): only eigenvector \(\begin{pmatrix}1\\0\end{pmatrix}\) — geometric multiplicity 1 < 2.

(c) Diagonalizable. Distinct eigenvalues 1, 2, 3 (triangular matrix).

\(P=\begin{pmatrix}2&0&0\\-2&1&0\\-1&0&1\end{pmatrix}\), \(D=\text{diag}(1,2,3)\).

(d) Not diagonalizable. Eigenvalue 2 with algebraic multiplicity 3. \((D-2I)\) is upper triangular with two 1s above the diagonal: rank 2, null space dimension 1. Geometric multiplicity 1 < 3.

4.26. State and Prove the Diagonalization Theorem (Assignment 7, Task 4.2)

State and prove: \(A\) is diagonalizable \(\iff\) it has \(n\) linearly independent eigenvectors. Explain why: (a) distinct eigenvalues \(\Rightarrow\) diagonalizable; (b) repeated eigenvalues can still be diagonalizable; (c) repeated eigenvalues may prevent diagonalization.

Click to see the solution

Theorem: \(A\in\mathbb{R}^{n\times n}\) is diagonalizable iff it has \(n\) linearly independent eigenvectors.

Proof (\(\Rightarrow\)): If \(P^{-1}AP=D\) with \(D\) diagonal, then \(AP=PD\), meaning each column \(\mathbf{p}_i\) of \(P\) satisfies \(A\mathbf{p}_i=d_{ii}\mathbf{p}_i\). So the \(n\) columns of \(P\) are eigenvectors; since \(P\) is invertible they are linearly independent. \(\square\)

Proof (\(\Leftarrow\)): If \(\mathbf{v}_1,\ldots,\mathbf{v}_n\) are \(n\) linearly independent eigenvectors with eigenvalues \(\lambda_1,\ldots,\lambda_n\), let \(P=[\mathbf{v}_1\cdots\mathbf{v}_n]\) (invertible). Then \(AP=PD\) where \(D=\text{diag}(\lambda_1,\ldots,\lambda_n)\), so \(P^{-1}AP=D\). \(\square\)

(a) Eigenvectors for distinct eigenvalues are always linearly independent (by the standard theorem), giving the needed \(n\) vectors automatically.

(b) If a repeated eigenvalue has a full eigenspace (e.g., the identity \(I\): \(\lambda=1\) mult. \(n\), eigenspace = all of \(\mathbb{R}^n\)), there are still \(n\) independent eigenvectors.

(c) Example: \(B=\begin{pmatrix}2&1\\0&2\end{pmatrix}\) has \(\lambda=2\) (mult. 2) but eigenspace of dimension 1 — cannot find 2 independent eigenvectors.

4.27. Compute \(A^5\) Using Diagonalization (Assignment 7, Task 4.3)

\(A\) is \(3\times3\) with eigenvalues \(\lambda=2,2,3\) and eigenvectors \(\mathbf{v}_1=\begin{pmatrix}1\\1\\0\end{pmatrix}\), \(\mathbf{v}_2=\begin{pmatrix}1\\0\\1\end{pmatrix}\), \(\mathbf{v}_3=\begin{pmatrix}0\\1\\1\end{pmatrix}\).

(a) Is \(A\) diagonalizable? (b) Find \(P\) and \(D\). (c) Compute \(A^5\).

Click to see the solution

(a) Three linearly independent eigenvectors for a \(3\times3\) matrix: yes, diagonalizable.

(b)

\[P=\begin{pmatrix}1&1&0\\1&0&1\\0&1&1\end{pmatrix}, \quad D=\begin{pmatrix}2&0&0\\0&2&0\\0&0&3\end{pmatrix}\]

(c) \(A^5=PD^5P^{-1}\). Find \(P^{-1}\): \(\det(P)=1(-1)-1(1)=-2\).

\[P^{-1}=-\frac{1}{2}\begin{pmatrix}-1&1&1\\1&-1&1\\1&1&-1\end{pmatrix}\]

\[D^5=\begin{pmatrix}32&0&0\\0&32&0\\0&0&243\end{pmatrix}\]

\[A^5=-\frac{1}{2}\begin{pmatrix}1&1&0\\1&0&1\\0&1&1\end{pmatrix}\begin{pmatrix}32&0&0\\0&32&0\\0&0&243\end{pmatrix}\begin{pmatrix}-1&1&1\\1&-1&1\\1&1&-1\end{pmatrix}\]

Answer: \(A^5 = PD^5P^{-1}\) as above (\(D^5 = \text{diag}(32,32,243)\)).

4.28. Orthogonal Diagonalization of a Symmetric Matrix (Assignment 7, Task 4.4)

Let \(A=\begin{pmatrix}4&2\\2&4\end{pmatrix}\).

(a) Find eigenvalues and eigenvectors. (b) Verify eigenvectors for distinct eigenvalues are orthogonal. (c) Find an orthogonal \(Q\) (\(Q^T=Q^{-1}\)) that diagonalizes \(A\). (d) What special property must \(A\) have for orthogonal diagonalization?

Click to see the solution

(a) \(\det(A-\lambda I)=(4-\lambda)^2-4=\lambda^2-8\lambda+12=(\lambda-6)(\lambda-2)=0\).

\(\lambda=6\): \(\begin{pmatrix}-2&2\\2&-2\end{pmatrix}\mathbf{v}=\mathbf{0}\Rightarrow v_1=v_2\), eigenvector \(\begin{pmatrix}1\\1\end{pmatrix}\).

\(\lambda=2\): \(\begin{pmatrix}2&2\\2&2\end{pmatrix}\mathbf{v}=\mathbf{0}\Rightarrow v_1=-v_2\), eigenvector \(\begin{pmatrix}1\\-1\end{pmatrix}\).

(b) \(\begin{pmatrix}1\\1\end{pmatrix}\cdot\begin{pmatrix}1\\-1\end{pmatrix}=1-1=0\) ✓ — orthogonal.

(c) Normalize: \(\mathbf{q}_1=\frac{1}{\sqrt{2}}\begin{pmatrix}1\\1\end{pmatrix}\), \(\mathbf{q}_2=\frac{1}{\sqrt{2}}\begin{pmatrix}1\\-1\end{pmatrix}\).

\[Q=\frac{1}{\sqrt{2}}\begin{pmatrix}1&1\\1&-1\end{pmatrix}, \quad Q^TAQ=\begin{pmatrix}6&0\\0&2\end{pmatrix}\]

(d) A real matrix is orthogonally diagonalizable if and only if it is symmetric (\(A=A^T\)). Over \(\mathbb{C}\), the analogous condition is Hermitian (\(A=A^H\)), giving unitary diagonalization. This is the content of the Spectral Theorem.

4.29. Constructing Matrices with Given Spectral Properties (Assignment 7, Task 4.5)

For each case, find such a matrix or explain why it is impossible:

(a) \(2\times2\) with eigenvalues 2 and 3 that is NOT diagonalizable.

(b) \(3\times3\) with eigenvalues 1, 1, 2 that IS diagonalizable.

(c) \(3\times3\) with eigenvalues 1, 1, 2 that is NOT diagonalizable.

(d) \(2\times2\) with no real eigenvalues, diagonalizable over \(\mathbb{C}\).

(e) \(2\times2\) with no real eigenvalues, diagonalizable over \(\mathbb{R}\).

Click to see the solution

(a) Impossible. Distinct eigenvalues always produce linearly independent eigenvectors, guaranteeing diagonalizability. There is no \(2\times2\) matrix with distinct eigenvalues that fails to be diagonalizable.

(b) Possible. \(A=\begin{pmatrix}1&0&0\\0&1&0\\0&0&2\end{pmatrix}\) has eigenvalues 1, 1, 2 and is already diagonal (diagonalizable).

(c) Possible. \(A=\begin{pmatrix}1&1&0\\0&1&0\\0&0&2\end{pmatrix}\): eigenvalue 1 (mult. 2) has only one independent eigenvector \(\begin{pmatrix}1\\0\\0\end{pmatrix}\) (geometric mult. 1 < 2). Not diagonalizable.

(d) Possible. Any real matrix with complex eigenvalues, e.g. rotation \(R=\begin{pmatrix}0&-1\\1&0\end{pmatrix}\) with \(\lambda=\pm i\). Over \(\mathbb{C}\), distinct eigenvalues \(\Rightarrow\) diagonalizable.

(e) Impossible. Diagonalization over \(\mathbb{R}\) requires real eigenvalues (since \(D\) must have real entries). A \(2\times2\) real matrix with no real eigenvalues cannot be diagonalized over \(\mathbb{R}\).

4.30. Polynomial of a Diagonalizable Matrix (Assignment 7, Task 4.6)

Suppose \(A\) is diagonalizable and \(p(t)\) is a polynomial.

(a) Show that \(p(A)\) is also diagonalizable.

(b) If \(A\) has eigenvalues \(\lambda_1,\ldots,\lambda_n\), what are the eigenvalues of \(p(A)\)?

(c) Use this to compute \(A^3-2A+I\) for \(A=\begin{pmatrix}2&1\\0&3\end{pmatrix}\) without matrix multiplication.

Click to see the solution

(a) If \(A=PDP^{-1}\), then \(p(A)=Pp(D)P^{-1}\). Since \(D=\text{diag}(\lambda_1,\ldots,\lambda_n)\), we have \(p(D)=\text{diag}(p(\lambda_1),\ldots,p(\lambda_n))\) — diagonal. So \(p(A)\) is diagonalizable (same \(P\)). \(\square\)

(b) The eigenvalues of \(p(A)\) are \(p(\lambda_1),\ldots,p(\lambda_n)\).

(c) \(A\) is upper triangular with eigenvalues \(\lambda_1=2\), \(\lambda_2=3\).

\[p(\lambda_1)=2^3-2(2)+1=8-4+1=5, \quad p(\lambda_2)=3^3-2(3)+1=27-6+1=22\]

So \(p(A)\) has eigenvalues 5 and 22, and is similar to \(\begin{pmatrix}5&0\\0&22\end{pmatrix}\).

Answer: (a) proved; (b) \(p(\lambda_i)\); (c) \(A^3-2A+I\) has eigenvalues 5 and 22.

4.31. Simultaneous Diagonalization (Assignment 7, Task 5.1)

Prove that if \(A\) and \(B\) are both diagonalizable and commute (\(AB=BA\)), then there exists an invertible \(P\) such that \(P^{-1}AP\) and \(P^{-1}BP\) are both diagonal (simultaneously diagonalizable).

Click to see the solution

Key Concept: The proof uses the fact that \(B\) maps eigenspaces of \(A\) to themselves, then diagonalizes \(B\) within each eigenspace.

Proof sketch:

1. Diagonalize \(A\): let \(P^{-1}AP = D_A\) with distinct eigenvalues on the diagonal. Decompose \(\mathbb{R}^n = E_{\lambda_1} \oplus \cdots \oplus E_{\lambda_k}\) (eigenspace decomposition).
2. For each eigenspace \(E_{\lambda_i}\): if \(A\mathbf{v}=\lambda_i\mathbf{v}\) and \(AB=BA\), then \(A(B\mathbf{v})=B(A\mathbf{v})=\lambda_i(B\mathbf{v})\), so \(B\mathbf{v}\in E_{\lambda_i}\). That is, \(B\) maps each eigenspace of \(A\) to itself.
3. Within each \(E_{\lambda_i}\), \(B\) is diagonalizable (since \(B\) is diagonalizable overall). Diagonalize \(B\) restricted to \(E_{\lambda_i}\) using an orthonormal basis.
4. Assembling all these bases gives a single invertible matrix \(P\) that diagonalizes both \(A\) and \(B\). \(\square\)

4.32. Spectral Decomposition (Assignment 7, Task 5.2)

Let \(A\) be \(n\times n\) with \(n\) linearly independent eigenvectors \(\mathbf{v}_1,\ldots,\mathbf{v}_n\) and eigenvalues \(\lambda_1,\ldots,\lambda_n\). Let \(\mathbf{w}_1,\ldots,\mathbf{w}_n\) be the rows of \(P^{-1}\) (the left eigenvectors).

(a) Show \(A = \sum_{i=1}^n \lambda_i\mathbf{v}_i\mathbf{w}_i^H\).

(b) What are the \(\mathbf{w}_i\) in terms of eigenvectors of \(A^H\)?

(c) Verify for \(A=\begin{pmatrix}2&1\\1&2\end{pmatrix}\).

Click to see the solution

(a) From \(A=PDP^{-1}\): write \(P=[\mathbf{v}_1\cdots\mathbf{v}_n]\) and \(P^{-1}\) has rows \(\mathbf{w}_1^H,\ldots,\mathbf{w}_n^H\) (left eigenvectors). Then:

\[A = PDP^{-1} = \sum_{i=1}^n\lambda_i\mathbf{v}_i\mathbf{w}_i^H \quad \square\]

This is the spectral decomposition of \(A\).

(b) The row vectors \(\mathbf{w}_i\) satisfy \(\mathbf{w}_i^H A = \lambda_i\mathbf{w}_i^H\), i.e., \(A^H\mathbf{w}_i=\overline{\lambda_i}\mathbf{w}_i\). So \(\mathbf{w}_i\) are eigenvectors of \(A^H\) with eigenvalues \(\overline{\lambda_i}\).

(c) \(A\) is symmetric: \(\lambda_1=3\), \(\mathbf{v}_1=\frac{1}{\sqrt{2}}\begin{pmatrix}1\\1\end{pmatrix}\); \(\lambda_2=1\), \(\mathbf{v}_2=\frac{1}{\sqrt{2}}\begin{pmatrix}1\\-1\end{pmatrix}\). Since \(A\) is symmetric, \(\mathbf{w}_i=\mathbf{v}_i\):

\[A = 3\mathbf{v}_1\mathbf{v}_1^T + 1\mathbf{v}_2\mathbf{v}_2^T = \frac{3}{2}\begin{pmatrix}1&1\\1&1\end{pmatrix}+\frac{1}{2}\begin{pmatrix}1&-1\\-1&1\end{pmatrix}=\begin{pmatrix}2&1\\1&2\end{pmatrix} \checkmark\]

4.33. The Cayley-Hamilton Theorem (Assignment 7, Task 5.3)

(a) Verify the Cayley-Hamilton theorem for \(A=\begin{pmatrix}2&1\\0&2\end{pmatrix}\).

(b) Use Cayley-Hamilton to find \(A^{-1}\) for \(A=\begin{pmatrix}2&1\\1&2\end{pmatrix}\).

(c) For a diagonalizable matrix, explain why Cayley-Hamilton holds.

Click to see the solution

Cayley-Hamilton Theorem: Every square matrix satisfies its own characteristic equation: if \(p(\lambda)=\det(A-\lambda I)\), then \(p(A)=0\).

(a) For \(A=\begin{pmatrix}2&1\\0&2\end{pmatrix}\): characteristic polynomial \(p(\lambda)=(\lambda-2)^2=\lambda^2-4\lambda+4\).

\[p(A)=A^2-4A+4I=\begin{pmatrix}4&4\\0&4\end{pmatrix}-\begin{pmatrix}8&4\\0&8\end{pmatrix}+\begin{pmatrix}4&0\\0&4\end{pmatrix}=\begin{pmatrix}0&0\\0&0\end{pmatrix} \checkmark\]

(b) For \(A=\begin{pmatrix}2&1\\1&2\end{pmatrix}\): \(p(\lambda)=\lambda^2-4\lambda+3\), so \(p(A)=A^2-4A+3I=0\).

Rearrange: \(A^2-4A=-3I \Rightarrow A(A-4I)=-3I \Rightarrow A\cdot\frac{-(A-4I)}{3}=I\).

\[A^{-1}=\frac{4I-A}{3}=\frac{1}{3}\begin{pmatrix}2&-1\\-1&2\end{pmatrix}\]

Verify: \(A\cdot A^{-1}=\frac{1}{3}\begin{pmatrix}2&1\\1&2\end{pmatrix}\begin{pmatrix}2&-1\\-1&2\end{pmatrix}=\frac{1}{3}\begin{pmatrix}3&0\\0&3\end{pmatrix}=I\) ✓.

(c) If \(A=PDP^{-1}\), then \(p(A)=Pp(D)P^{-1}\). For each diagonal entry \(\lambda_i\), \(p(\lambda_i)=\det(A-\lambda_i I)=0\) (since \(\lambda_i\) is an eigenvalue). So \(p(D)=\text{diag}(p(\lambda_1),\ldots,p(\lambda_n))=0\), hence \(p(A)=P\cdot 0\cdot P^{-1}=0\). \(\square\)

4.34. Complex Vector Operations (Lecture 7, Example 1)

Let \(\mathbf{v} = \begin{bmatrix} 2 + i \\ -3i \end{bmatrix}\), \(\mathbf{w} = \begin{bmatrix} 1 - i \\ 2 \end{bmatrix}\), and \(\alpha = 2i\).

(a) Compute \(\mathbf{v} + \mathbf{w}\).

(b) Compute \(\alpha\mathbf{v}\).

Click to see the solution

Key Concept: Vector addition and scalar multiplication work component-wise. Arithmetic with complex numbers follows \(i^2 = -1\).

(a) Addition:

\[\mathbf{v} + \mathbf{w} = \begin{bmatrix} (2+i) + (1-i) \\ -3i + 2 \end{bmatrix} = \begin{bmatrix} 3 \\ 2 - 3i \end{bmatrix}\]

(b) Scalar multiplication by \(\alpha = 2i\):

\[\alpha\mathbf{v} = \begin{bmatrix} 2i(2+i) \\ 2i(-3i) \end{bmatrix} = \begin{bmatrix} 4i + 2i^2 \\ -6i^2 \end{bmatrix} = \begin{bmatrix} 4i - 2 \\ 6 \end{bmatrix} = \begin{bmatrix} -2 + 4i \\ 6 \end{bmatrix}\]

Note: \(i^2 = -1\), so \(2i^2 = -2\) and \(-6i^2 = 6\).

Answer:

1. \(\mathbf{v} + \mathbf{w} = \begin{bmatrix} 3 \\ 2 - 3i \end{bmatrix}\)

2. \(\alpha\mathbf{v} = \begin{bmatrix} -2 + 4i \\ 6 \end{bmatrix}\)

4.35. Norm of a Complex Vector (Lecture 7, Example 2)

Compute the norm of \(\mathbf{v} = \begin{bmatrix} 1 + i \\ 2 - i \end{bmatrix}\).

Click to see the solution

Key Concept: The norm uses the modulus of each component: \(\|\mathbf{v}\| = \sqrt{\sum |v_i|^2}\). Recall \(|a + bi|^2 = a^2 + b^2\).

1. Compute \(|v_1|^2\): \[|1 + i|^2 = 1^2 + 1^2 = 2\]
2. Compute \(|v_2|^2\): \[|2 - i|^2 = 2^2 + (-1)^2 = 4 + 1 = 5\]
3. Apply the norm formula: \[\|\mathbf{v}\| = \sqrt{|v_1|^2 + |v_2|^2} = \sqrt{2 + 5} = \sqrt{7}\]

Answer: \(\|\mathbf{v}\| = \sqrt{7}\)

4.36. Verify a Unitary Matrix and Find Its Eigenvalues (Lecture 7, Example 3)

Verify that \(U = \dfrac{1}{\sqrt{2}}\begin{bmatrix} 1 & i \\ i & 1 \end{bmatrix}\) is unitary and find its eigenvalues.

Click to see the solution

Key Concept: A matrix is unitary if \(U^H U = I\). Eigenvalues of unitary matrices must lie on the unit circle: \(|\lambda| = 1\).

Step 1: Compute \(U^H\)

\[U^H = \frac{1}{\sqrt{2}}\begin{bmatrix} \overline{1} & \overline{i} \\ \overline{i} & \overline{1} \end{bmatrix} = \frac{1}{\sqrt{2}}\begin{bmatrix} 1 & -i \\ -i & 1 \end{bmatrix}\]

Step 2: Verify \(U^H U = I\)

\[U^H U = \frac{1}{2}\begin{bmatrix} 1 & -i \\ -i & 1 \end{bmatrix}\begin{bmatrix} 1 & i \\ i & 1 \end{bmatrix} = \frac{1}{2}\begin{bmatrix} 1 + 1 & i - i \\ -i + i & 1 + 1 \end{bmatrix} = \frac{1}{2}\begin{bmatrix} 2 & 0 \\ 0 & 2 \end{bmatrix} = I \quad \checkmark\]

Step 3: Find eigenvalues

\[\det(U - \lambda I) = \det\begin{bmatrix} \frac{1}{\sqrt{2}} - \lambda & \frac{i}{\sqrt{2}} \\ \frac{i}{\sqrt{2}} & \frac{1}{\sqrt{2}} - \lambda \end{bmatrix} = \left(\frac{1}{\sqrt{2}} - \lambda\right)^2 - \frac{i^2}{2} = 0\]

\[\left(\frac{1}{\sqrt{2}} - \lambda\right)^2 + \frac{1}{2} = 0 \implies \lambda^2 - \sqrt{2}\,\lambda + 1 = 0\]

\[\lambda = \frac{\sqrt{2} \pm \sqrt{2 - 4}}{2} = \frac{\sqrt{2} \pm i\sqrt{2}}{2} = \frac{1 \pm i}{\sqrt{2}}\]

Step 4: Verify eigenvalues lie on unit circle

\[\left|\frac{1+i}{\sqrt{2}}\right| = \frac{|1+i|}{\sqrt{2}} = \frac{\sqrt{2}}{\sqrt{2}} = 1 \quad \checkmark\]

Answer: \(U\) is unitary. Its eigenvalues are \(\lambda = \dfrac{1+i}{\sqrt{2}}\) and \(\lambda = \dfrac{1-i}{\sqrt{2}}\), both with \(|\lambda| = 1\).

4.37. Operations on Complex Numbers (Lecture 7, Task 1)

Add and multiply each pair of complex numbers:

(a) \(2 + i\) and \(2 - i\)

(b) \(-1 + i\) and \(-1 + i\)

(c) \(\cos\theta + i\sin\theta\) and \(\cos\theta - i\sin\theta\)

Click to see the solution

Key Concept: Addition and multiplication follow standard algebra with \(i^2 = -1\). Part (c) also admits a shortcut using Euler’s formula \(e^{i\theta} = \cos\theta + i\sin\theta\).

(a) Let \(z_1 = 2 + i\), \(z_2 = 2 - i\):

\[z_1 + z_2 = 4, \quad z_1\cdot z_2 = (2+i)(2-i) = 4 - i^2 = 4 + 1 = 5\]

(b) Let \(z = -1 + i\):

\[z + z = -2 + 2i, \quad z \cdot z = (-1+i)^2 = 1 - 2i + i^2 = 1 - 2i - 1 = -2i\]

(c) Let \(z_1 = \cos\theta + i\sin\theta = e^{i\theta}\) and \(z_2 = \cos\theta - i\sin\theta = e^{-i\theta}\):

\[z_1 + z_2 = 2\cos\theta\]

For the product, either compute directly: \[z_1\cdot z_2 = (\cos\theta + i\sin\theta)(\cos\theta - i\sin\theta) = \cos^2\theta + \sin^2\theta = 1\]

or use exponentials: \(e^{i\theta}\cdot e^{-i\theta} = e^{0} = 1\).

Answer: (a) Sum: \(4\); Product: \(5\). (b) Sum: \(-2+2i\); Product: \(-2i\). (c) Sum: \(2\cos\theta\); Product: \(1\).

4.38. Properties of a Complex Number and Its Conjugate (Lecture 7, Task 2)

Let \(z = a + ib\). Express each of the following in simplest form:

(a) \(z + \bar{z}\)

(b) \(z - \bar{z}\)

(c) \(z \cdot \bar{z}\)

(d) \(z / \bar{z}\) (for \(\bar{z} \neq 0\)). What is the absolute value of \(z / \bar{z}\)?

Click to see the solution

Key Concept: The conjugate of \(z = a + ib\) is \(\bar{z} = a - ib\). Adding or multiplying \(z\) and \(\bar{z}\) produces real quantities.

1. (a) Sum: \[z + \bar{z} = (a + ib) + (a - ib) = 2a = 2\,\text{Re}(z)\]
2. (b) Difference: \[z - \bar{z} = (a + ib) - (a - ib) = 2ib = 2i\,\text{Im}(z)\]
3. (c) Product: \[z\bar{z} = (a + ib)(a - ib) = a^2 - (ib)^2 = a^2 + b^2 = |z|^2\] Always a non-negative real number.
4. (d) Ratio: Multiply numerator and denominator by \(z\): \[\frac{z}{\bar{z}} = \frac{z \cdot z}{\bar{z} \cdot z} = \frac{z^2}{|z|^2}\] Since \(\left|\dfrac{z}{\bar{z}}\right| = \dfrac{|z|}{|\bar{z}|} = \dfrac{|z|}{|z|} = 1\), the ratio always has absolute value 1. Geometrically, \(z/\bar{z}\) lies on the unit circle.

Answer: (a) \(2\,\text{Re}(z)\), (b) \(2i\,\text{Im}(z)\), (c) \(|z|^2\), (d) \(z^2/|z|^2\) with \(|z/\bar{z}| = 1\).

4.39. Norms and Inner Product of Complex Vectors (Lecture 7, Task 3)

Given \(\mathbf{u} = \begin{pmatrix} 1+i \\ 1-i \\ 1+2i \end{pmatrix}\) and \(\mathbf{v} = \begin{pmatrix} i \\ i \\ i \end{pmatrix}\), find:

(a) \(\|\mathbf{u}\|\) and \(\|\mathbf{v}\|\)

(b) \(\mathbf{u}^H\mathbf{v}\) and \(\mathbf{v}^H\mathbf{u}\)

Click to see the solution

Key Concept: \(\|\mathbf{v}\|^2 = \mathbf{v}^H\mathbf{v} = \sum |v_i|^2\). The inner product \(\mathbf{u}^H\mathbf{v}\) conjugates the entries of the first vector.

(a) Norms:

\[\|\mathbf{u}\|^2 = |1+i|^2 + |1-i|^2 + |1+2i|^2 = 2 + 2 + 5 = 9 \implies \|\mathbf{u}\| = 3\]

\[\|\mathbf{v}\|^2 = |i|^2 + |i|^2 + |i|^2 = 1 + 1 + 1 = 3 \implies \|\mathbf{v}\| = \sqrt{3}\]

(b) Inner products:

The conjugate of \(\mathbf{u}\) is \(\bar{\mathbf{u}} = \begin{pmatrix} 1-i \\ 1+i \\ 1-2i \end{pmatrix}\), so:

\[\mathbf{u}^H\mathbf{v} = (1-i)(i) + (1+i)(i) + (1-2i)(i)\] \[= (i - i^2) + (i + i^2) + (i - 2i^2) = (i+1) + (i-1) + (i+2) = 2 + 3i\]

\[\mathbf{v}^H\mathbf{u} = \overline{\mathbf{u}^H\mathbf{v}} = 2 - 3i\]

Note that \(\mathbf{v}^H\mathbf{u} = \overline{\mathbf{u}^H\mathbf{v}}\) — this is the conjugate symmetry property of the Hermitian inner product.

Answer: \(\|\mathbf{u}\| = 3\), \(\|\mathbf{v}\| = \sqrt{3}\), \(\mathbf{u}^H\mathbf{v} = 2 + 3i\), \(\mathbf{v}^H\mathbf{u} = 2 - 3i\).

4.40. Classify a Matrix and Compute Its Powers (Lecture 7, Task 4)

Consider \(P = \begin{pmatrix} 0 & i & 0 \\ 0 & 0 & i \\ i & 0 & 0 \end{pmatrix}\).

(a) Determine whether \(P\) is Hermitian, invertible, or unitary.

(b) Compute \(P^2\), \(P^3\), and \(P^{100}\).

Click to see the solution

Key Concept: Compute \(P^H\) to check Hermitian and unitary conditions. For powers, look for a pattern from \(P^2\) and \(P^3\).

(a) Classification:

First compute \(P^H\): transpose and conjugate all entries. The entries of \(P\) are all \(0\) or \(i\), so conjugates are \(0\) or \(-i\):

\[P^H = \begin{pmatrix} 0 & 0 & -i \\ -i & 0 & 0 \\ 0 & -i & 0 \end{pmatrix}\]

• Hermitian? \(P \neq P^H\), so No.
• Invertible? \(\det(P) = i\cdot i\cdot i \cdot(-1)^{\text{sign}} = -i \neq 0\) (expanding along first column), so Yes.
• Unitary? Check \(PP^H = I\): \[PP^H = \begin{pmatrix} 0 & i & 0 \\ 0 & 0 & i \\ i & 0 & 0 \end{pmatrix}\begin{pmatrix} 0 & 0 & -i \\ -i & 0 & 0 \\ 0 & -i & 0 \end{pmatrix} = \begin{pmatrix} -i^2 & 0 & 0 \\ 0 & -i^2 & 0 \\ 0 & 0 & -i^2 \end{pmatrix} = I \quad \checkmark\]

So \(P\) is unitary (and invertible), but not Hermitian.

(b) Powers:

\[P^2 = \begin{pmatrix} 0 & i & 0 \\ 0 & 0 & i \\ i & 0 & 0 \end{pmatrix}^2 = -\begin{pmatrix} 0 & 0 & 1 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \end{pmatrix}\]

\[P^3 = P^2 \cdot P = -\begin{pmatrix} 0 & 0 & 1 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \end{pmatrix}\begin{pmatrix} 0 & i & 0 \\ 0 & 0 & i \\ i & 0 & 0 \end{pmatrix} = -iI\]

So \(P^3 = -iI\). Therefore \((P^3)^{33} = (-i)^{33}I\).

For \(P^{100} = P^{99}\cdot P = (P^3)^{33}\cdot P\):

\[(-i)^{33} = (-1)^{33}\cdot i^{33} = (-1)\cdot i^{32}\cdot i = (-1)\cdot(i^4)^8\cdot i = (-1)\cdot 1\cdot i = -i\]

\[P^{100} = (-i)I \cdot P = -iP\]

Answer: \(P\) is unitary but not Hermitian. \(P^3 = -iI\), and \(P^{100} = -iP\).

4.41. Prove a Formula for Matrix Powers Using Diagonalization (Lecture 7, Task 5)

Let \(B = \begin{pmatrix} 3 & 1 \\ 0 & 2 \end{pmatrix}\). Diagonalize \(B\) to prove that:

\[B^k = \begin{pmatrix} 3^k & 3^k - 2^k \\ 0 & 2^k \end{pmatrix}\]

Click to see the solution

Key Concept: Diagonalization gives \(B^k = S\Lambda^k S^{-1}\). Since \(B\) is upper triangular, its eigenvalues are the diagonal entries.

Step 1: Find eigenvalues

Eigenvalues of \(B\) are its diagonal entries: \(\lambda_1 = 3\), \(\lambda_2 = 2\) (distinct, so \(B\) is diagonalizable).

Check: \(\text{tr}(B) = 5 = 3 + 2\) ✓ and \(\det(B) = 6 = 3 \cdot 2\) ✓.

Step 2: Find eigenvectors

For \(\lambda_1 = 3\): \((B - 3I)\mathbf{v} = \begin{pmatrix} 0 & 1 \\ 0 & -1 \end{pmatrix}\mathbf{v} = \mathbf{0} \implies y = 0\), so \(\mathbf{v}_1 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}\).

For \(\lambda_2 = 2\): \((B - 2I)\mathbf{v} = \begin{pmatrix} 1 & 1 \\ 0 & 0 \end{pmatrix}\mathbf{v} = \mathbf{0} \implies x + y = 0\), so \(\mathbf{v}_2 = \begin{pmatrix} -1 \\ 1 \end{pmatrix}\).

Step 3: Write the diagonalization

\[S = \begin{pmatrix} 1 & -1 \\ 0 & 1 \end{pmatrix}, \quad \Lambda = \begin{pmatrix} 3 & 0 \\ 0 & 2 \end{pmatrix}, \quad S^{-1} = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}\]

Step 4: Compute \(B^k = S\Lambda^k S^{-1}\)

\[B^k = \begin{pmatrix} 1 & -1 \\ 0 & 1 \end{pmatrix}\begin{pmatrix} 3^k & 0 \\ 0 & 2^k \end{pmatrix}\begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} 3^k & -2^k \\ 0 & 2^k \end{pmatrix}\begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} 3^k & 3^k - 2^k \\ 0 & 2^k \end{pmatrix} \quad \checkmark\]

Answer: \(B^k = \begin{pmatrix} 3^k & 3^k - 2^k \\ 0 & 2^k \end{pmatrix}\).

4.42. Reconstruct a Matrix from Trace and Determinant Conditions (Lecture 7, Task 6)

Choose the second row of \(A\) so that \(A\) has eigenvalues \(4\) and \(7\): \[A = \begin{pmatrix} 1 & 1 \\ \square & \square \end{pmatrix}\]

Click to see the solution

Key Concept: Use the Vieta’s formulas relating eigenvalues to the trace and determinant: \(\lambda_1 + \lambda_2 = \text{tr}(A)\) and \(\lambda_1\lambda_2 = \det(A)\).

Let the second row be \((a,\, b)\). Then:

• \(\text{tr}(A) = 1 + b\) and \(\det(A) = b - a\)

From the required eigenvalues \(\lambda_1 = 4\), \(\lambda_2 = 7\):

• \(\lambda_1 + \lambda_2 = 11 \implies 1 + b = 11 \implies b = 10\)
• \(\lambda_1 \cdot \lambda_2 = 28 \implies b - a = 28 \implies 10 - a = 28 \implies a = -18\)

Verification: With \(A = \begin{pmatrix} 1 & 1 \\ -18 & 10 \end{pmatrix}\), the characteristic polynomial is: \[\lambda^2 - 11\lambda + (10 - (-18)) = \lambda^2 - 11\lambda + 28 = (\lambda - 4)(\lambda - 7) = 0 \quad \checkmark\]

Answer: The second row is \((-18,\, 10)\), giving \(A = \begin{pmatrix} 1 & 1 \\ -18 & 10 \end{pmatrix}\).

4.43. Reconstruct a Matrix from Its Eigendecomposition (Lecture 7, Task 7)

Find the matrix \(A\) whose eigenvalues are \(\lambda_1 = 1\) and \(\lambda_2 = 4\) with corresponding eigenvectors \((3, 1)^T\) and \((2, 1)^T\).

Click to see the solution

Key Concept: If \(A = S\Lambda S^{-1}\) where \(S\) has eigenvectors as columns and \(\Lambda\) has eigenvalues on the diagonal, we can reconstruct \(A\) by computing this product.

Step 1: Set up \(S\) and \(\Lambda\)

\[S = \begin{pmatrix} 3 & 2 \\ 1 & 1 \end{pmatrix}, \quad \Lambda = \begin{pmatrix} 1 & 0 \\ 0 & 4 \end{pmatrix}\]

Step 2: Compute \(S^{-1}\)

\[\det(S) = 3\cdot 1 - 2\cdot 1 = 1, \quad S^{-1} = \begin{pmatrix} 1 & -2 \\ -1 & 3 \end{pmatrix}\]

Step 3: Compute \(A = S\Lambda S^{-1}\)

\[A = \begin{pmatrix} 3 & 2 \\ 1 & 1 \end{pmatrix}\begin{pmatrix} 1 & 0 \\ 0 & 4 \end{pmatrix}\begin{pmatrix} 1 & -2 \\ -1 & 3 \end{pmatrix} = \begin{pmatrix} 3 & 8 \\ 1 & 4 \end{pmatrix}\begin{pmatrix} 1 & -2 \\ -1 & 3 \end{pmatrix} = \begin{pmatrix} -5 & 18 \\ -3 & 10 \end{pmatrix}\]

Verification: \(A\begin{pmatrix} 3 \\ 1 \end{pmatrix} = \begin{pmatrix} -15+18 \\ -9+10 \end{pmatrix} = \begin{pmatrix} 3 \\ 1 \end{pmatrix} = 1\cdot\begin{pmatrix} 3 \\ 1 \end{pmatrix}\) ✓

Answer: \(A = \begin{pmatrix} -5 & 18 \\ -3 & 10 \end{pmatrix}\).

4.44. Complex Numbers in Polar Form (Lecture 7, Task 8)

If \(x = re^{i\theta}\), find \(x^2\), \(x^{-1}\), and \(\bar{x}\) in polar coordinates. For which complex numbers does \(x^{-1} = \bar{x}\)?

Click to see the solution

Key Concept: In polar form \(z = re^{i\theta}\), multiplication of complex numbers adds arguments and multiplies moduli.

1. \(x^2\): \((re^{i\theta})^2 = r^2 e^{2i\theta}\) — modulus squared, argument doubled.
2. \(x^{-1}\): \(\dfrac{1}{re^{i\theta}} = r^{-1}e^{-i\theta}\) — reciprocal modulus, negated argument.
3. \(\bar{x}\): The conjugate negates the imaginary part, which in polar form means negating the argument while keeping the modulus: \(\bar{x} = re^{-i\theta}\).

When is \(x^{-1} = \bar{x}\)?

\[r^{-1}e^{-i\theta} = re^{-i\theta} \implies r^{-1} = r \implies r^2 = 1 \implies r = 1\]

So \(x^{-1} = \bar{x}\) precisely when \(|x| = 1\), i.e., when \(x\) lies on the unit circle in the complex plane. This is exactly the condition defining unitary (orthogonal) scalars.

Answer: \(x^2 = r^2e^{2i\theta}\); \(x^{-1} = r^{-1}e^{-i\theta}\); \(\bar{x} = re^{-i\theta}\). The condition \(x^{-1} = \bar{x}\) holds if and only if \(|x| = 1\).

4.45. Compute \(A^H\) and Verify Hermitian Property (Lecture 7, Task 9)

Let \(A = \begin{bmatrix} 1 & i & 0 \\ i & 0 & 1 \end{bmatrix}\).

(a) Write out \(A^H\) and compute \(C = A^H A\).

(b) What is the relationship between \(C\) and \(C^H\)?

Click to see the solution

Key Concept: \(A^H\) is the conjugate transpose. The product \(A^H A\) is always Hermitian (\(C = C^H\)) and positive semidefinite.

(a) Computing \(A^H\):

Transpose \(A\), then conjugate every entry: \[A^H = \begin{bmatrix} \bar{1} & \bar{i} \\ \bar{i} & \bar{0} \\ \bar{0} & \bar{1} \end{bmatrix} = \begin{bmatrix} 1 & -i \\ -i & 0 \\ 0 & 1 \end{bmatrix}\]

Computing \(C = A^H A\):

\[C = \begin{bmatrix} 1 & -i \\ -i & 0 \\ 0 & 1 \end{bmatrix}\begin{bmatrix} 1 & i & 0 \\ i & 0 & 1 \end{bmatrix}\]

Row 1: \((1\cdot1 + (-i)\cdot i,\ 1\cdot i + (-i)\cdot 0,\ 1\cdot 0 + (-i)\cdot 1) = (1+1,\ i,\ -i) = (2,\ i,\ -i)\)

Row 2: \(((-i)\cdot1 + 0\cdot i,\ (-i)\cdot i + 0,\ (-i)\cdot 0 + 0) = (-i,\ 1,\ 0)\)

Row 3: \((0\cdot1 + 1\cdot i,\ 0\cdot i + 0,\ 0 + 1\cdot1) = (i,\ 0,\ 1)\)

\[C = \begin{bmatrix} 2 & i & -i \\ -i & 1 & 0 \\ i & 0 & 1 \end{bmatrix}\]

(b) Relationship between \(C\) and \(C^H\):

\(C^H = \overline{C^T}\). Check off-diagonal: \(C_{12} = i\), \(C_{21} = -i = \bar{i}\) ✓; \(C_{13} = -i\), \(C_{31} = i = \overline{-i}\) ✓. All diagonal entries \(2, 1, 1\) are real ✓.

Therefore \(C = C^H\): \(C\) is Hermitian.

This is always true: for any matrix \(A\), the product \(A^H A\) is Hermitian because \((A^H A)^H = A^H (A^H)^H = A^H A\).

Answer: \(A^H = \begin{bmatrix} 1 & -i \\ -i & 0 \\ 0 & 1 \end{bmatrix}\), \(C = \begin{bmatrix} 2 & i & -i \\ -i & 1 & 0 \\ i & 0 & 1 \end{bmatrix}\), and \(C = C^H\) (Hermitian).

4.46. Null Space and Complex Orthogonality (Lecture 7, Task 10)

For \(A = \begin{bmatrix} 1 & i & 0 \\ i & 0 & 1 \end{bmatrix}\):

(a) Use elimination to solve \(A\mathbf{x} = \mathbf{0}\).

(b) Show that the nullspace \(\mathcal{N}(A)\) is orthogonal to the column space of \(A^H\) (i.e., \(\mathcal{R}(A^H)\)), but not to \(\mathcal{R}(A^T)\).

Click to see the solution

Key Concept: In the complex case, the fundamental orthogonality is \(\mathcal{N}(A) \perp \mathcal{R}(A^H)\). The ordinary transpose \(A^T\) does not give the correct orthogonal complement.

(a) Solving \(A\mathbf{x} = \mathbf{0}\):

\[\begin{cases} x_1 + i\,x_2 = 0 \\ i\,x_1 + x_3 = 0 \end{cases}\]

From the first equation: \(x_1 = -i\,x_2\). Substitute into the second:

\[i(-i\,x_2) + x_3 = 0 \implies x_2 + x_3 = 0 \implies x_3 = -x_2\]

With \(x_2 = 1\) as the free variable:

\[\mathcal{N}(A) = \text{span}\left\{\begin{bmatrix} -i \\ 1 \\ -1 \end{bmatrix}\right\}\]

(b) Orthogonality:

Let \(\mathbf{n} = \begin{bmatrix} -i \\ 1 \\ -1 \end{bmatrix}\), so \(\mathbf{n}^H = \begin{bmatrix} i & 1 & -1 \end{bmatrix}\).

The columns of \(A^H = \begin{bmatrix} 1 & -i \\ -i & 0 \\ 0 & 1 \end{bmatrix}\) are \(\mathbf{c}_1 = \begin{bmatrix} 1 \\ -i \\ 0 \end{bmatrix}\) and \(\mathbf{c}_2 = \begin{bmatrix} -i \\ 0 \\ 1 \end{bmatrix}\).

Check \(\mathbf{n}^H\mathbf{c}_1 = i\cdot1 + 1\cdot(-i) + (-1)\cdot0 = i - i = 0\) ✓

Check \(\mathbf{n}^H\mathbf{c}_2 = i\cdot(-i) + 1\cdot0 + (-1)\cdot1 = 1 - 1 = 0\) ✓

So \(\mathcal{N}(A) \perp \mathcal{R}(A^H)\) ✓.

Now check against \(\mathcal{R}(A^T)\). The columns of \(A^T = \begin{bmatrix} 1 & i \\ i & 0 \\ 0 & 1 \end{bmatrix}\) include \(\mathbf{r}_1 = \begin{bmatrix} 1 \\ i \\ 0 \end{bmatrix}\):

\[\mathbf{n}^H\mathbf{r}_1 = i\cdot1 + 1\cdot i + (-1)\cdot0 = 2i \neq 0\]

So \(\mathcal{N}(A)\) is not orthogonal to \(\mathcal{R}(A^T)\).

Answer: The null space vector \(\mathbf{n} = (-i, 1, -1)^T\) is orthogonal to both columns of \(A^H\) but not to the columns of \(A^T\). In the complex case, the correct orthogonality uses \(A^H\), not \(A^T\).

4.47. Show Matrices Are Similar (Lecture 7, Task 11)

Show that each pair of matrices is similar by finding \(M\) such that \(B = M^{-1}AM\):

(a) \(A = \begin{bmatrix} 1 & 0 \\ 1 & 0 \end{bmatrix}\) and \(B = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix}\)

(b) \(A = \begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}\) and \(B = \begin{bmatrix} 1 & -1 \\ -1 & 1 \end{bmatrix}\)

(c) \(A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}\) and \(B = \begin{bmatrix} 4 & 3 \\ 2 & 1 \end{bmatrix}\)

Click to see the solution

Key Concept: To find \(M\) such that \(AM = MB\) (equivalently \(B = M^{-1}AM\)), set up the equation entry by entry and solve for the unknowns.

(a) Let \(M = \begin{bmatrix} a & b \\ c & d \end{bmatrix}\). We need \(AM = MB\):

\[AM = \begin{bmatrix} a & b \\ a & b \end{bmatrix}, \quad MB = \begin{bmatrix} a & 0 \\ c & 0 \end{bmatrix}\]

Equating: \(b = 0\) and \(a = c\). Choose \(a = 1, d = 1\): \(M = \begin{bmatrix} 1 & 0 \\ 1 & 1 \end{bmatrix}\), \(M^{-1} = \begin{bmatrix} 1 & 0 \\ -1 & 1 \end{bmatrix}\).

Verify: \(M^{-1}AM = \begin{bmatrix} 1 & 0 \\ -1 & 1 \end{bmatrix}\begin{bmatrix} 1 & 0 \\ 1 & 0 \end{bmatrix}\begin{bmatrix} 1 & 0 \\ 1 & 1 \end{bmatrix} = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix} = B\) ✓

(b) Let \(M = \begin{bmatrix} a & b \\ c & d \end{bmatrix}\). Setting \(AM = MB\):

\[AM = \begin{bmatrix} a+c & b+d \\ a+c & b+d \end{bmatrix}, \quad MB = \begin{bmatrix} a-b & -a+b \\ c-d & -c+d \end{bmatrix}\]

From columns: \(a + c = a - b \Rightarrow c = -b\); \(b + d = -a + b \Rightarrow d = -a\). Choose \(a = 1, b = 0\): \(M = \begin{bmatrix} 1 & 0 \\ 0 & -1 \end{bmatrix} = M^{-1}\).

Verify: \(M^{-1}AM = \begin{bmatrix} 1 & -1 \\ -1 & 1 \end{bmatrix} = B\) ✓

(c) Observe that \(B\) is obtained from \(A\) by reversing the order of both rows and columns. The permutation matrix \(P = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}\) satisfies \(P = P^{-1}\) and \(B = P^{-1}AP\). So \(M = P\).

Verify: \(PAP = \begin{bmatrix} 3 & 4 \\ 1 & 2 \end{bmatrix}\begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} = \begin{bmatrix} 4 & 3 \\ 2 & 1 \end{bmatrix} = B\) ✓

Answer: \(M_a = \begin{bmatrix} 1 & 0 \\ 1 & 1 \end{bmatrix}\); \(M_b = \begin{bmatrix} 1 & 0 \\ 0 & -1 \end{bmatrix}\); \(M_c = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}\).

4.48. Orthonormal Basis and Projection Matrix (Midterm Recap, Task 1)

Given vectors \(u = [1, 1, 1]^T\) and \(v = [1, 2, 3]^T\) forming matrix \(A = [u \ v]\):

(a) Find an orthonormal basis for the column space \(Col(A)\).

(b) Find the projection matrix \(P = A(A^TA)^{-1}A^T\).

(c) Find the projection of vector \(b = [0, 0, 1]^T\) onto \(Col(A)\).

Click to see the solution

Key Concept: Use the Gram-Schmidt process to orthonormalize the columns, then build the projection matrix.

(a) Apply the Gram-Schmidt process:

1. Normalize \(u\): \[\bar{u} = u = [1, 1, 1]^T \Rightarrow q_1 = \frac{1}{\sqrt{3}}[1, 1, 1]^T\]
2. Subtract the projection of \(v\) onto \(\bar{u}\): \[\bar{v} = v - \frac{v^T\bar{u}}{\bar{u}^T\bar{u}}\bar{u} = \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix} - \frac{6}{3}\begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix} = \begin{bmatrix} -1 \\ 0 \\ 1 \end{bmatrix}\]
3. Normalize \(\bar{v}\): \(q_2 = \frac{1}{\sqrt{2}}[-1, 0, 1]^T\)

Answer (a): Orthonormal basis for \(Col(A) = \text{span}\left\{ \begin{bmatrix} 1/\sqrt{3} \\ 1/\sqrt{3} \\ 1/\sqrt{3} \end{bmatrix},\ \begin{bmatrix} -1/\sqrt{2} \\ 0 \\ 1/\sqrt{2} \end{bmatrix} \right\}\)

(b) Compute \(A^TA\): \[A^TA = \begin{bmatrix} 3 & 6 \\ 6 & 14 \end{bmatrix}\]

Find its inverse: \(\det = 42 - 36 = 6\), so \((A^TA)^{-1} = \frac{1}{6}\begin{bmatrix} 14 & -6 \\ -6 & 3 \end{bmatrix}\)

Compute \(P\): \[P = \frac{1}{6}\begin{bmatrix} 1 & 1 \\ 1 & 2 \\ 1 & 3 \end{bmatrix}\begin{bmatrix} 14 & -6 \\ -6 & 3 \end{bmatrix}\begin{bmatrix} 1 & 1 & 1 \\ 1 & 2 & 3 \end{bmatrix} = \frac{1}{6}\begin{bmatrix} 5 & 2 & -1 \\ 2 & 2 & 2 \\ -1 & 2 & 5 \end{bmatrix}\]

Answer (b): \(P = \dfrac{1}{6}\begin{bmatrix} 5 & 2 & -1 \\ 2 & 2 & 2 \\ -1 & 2 & 5 \end{bmatrix}\)

(c) Apply \(P\) to \(b\): \[Pb = \frac{1}{6}\begin{bmatrix} 5 & 2 & -1 \\ 2 & 2 & 2 \\ -1 & 2 & 5 \end{bmatrix}\begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} = \frac{1}{6}\begin{bmatrix} -1 \\ 2 \\ 5 \end{bmatrix}\]

Answer (c): \(\dfrac{1}{6}\begin{bmatrix} -1 \\ 2 \\ 5 \end{bmatrix}\)

4.49. Least Squares Line Fitting (Midterm Recap, Task 2)

Given experimental data points \((t, y)\): \((1,1), (2,3), (3,4), (4,4)\). Fit a line \(y = C + Dt\) using the method of least squares.

(a) Set up the overdetermined system \(Ax = b\).

(b) Solve the normal equation \(A^TAx = A^Tb\) to find coefficients \(C\) and \(D\).

(c) Predict \(y(5)\).

Click to see the solution

Key Concept: For an overdetermined system with no exact solution, the least squares solution minimizes \(\|Ax - b\|^2\) and satisfies the normal equations \(A^TA\hat{x} = A^Tb\).

(a) Substituting each point into \(y = C + Dt\):

\[\begin{bmatrix} 1 & 1 \\ 1 & 2 \\ 1 & 3 \\ 1 & 4 \end{bmatrix} \begin{bmatrix} C \\ D \end{bmatrix} = \begin{bmatrix} 1 \\ 3 \\ 4 \\ 4 \end{bmatrix}\]

(b) Compute \(A^TA\) and \(A^Tb\): \[A^TA = \begin{bmatrix} 4 & 10 \\ 10 & 30 \end{bmatrix}, \quad A^Tb = \begin{bmatrix} 12 \\ 35 \end{bmatrix}\]

Solve: \(\det(A^TA) = 120 - 100 = 20\), so \((A^TA)^{-1} = \frac{1}{20}\begin{bmatrix} 30 & -10 \\ -10 & 4 \end{bmatrix}\)

\[\begin{bmatrix} C \\ D \end{bmatrix} = \frac{1}{20}\begin{bmatrix} 30 & -10 \\ -10 & 4 \end{bmatrix}\begin{bmatrix} 12 \\ 35 \end{bmatrix} = \frac{1}{20}\begin{bmatrix} 10 \\ 20 \end{bmatrix} = \begin{bmatrix} 0.5 \\ 1 \end{bmatrix}\]

Best-fit line: \(y = 0.5 + t\)

(c) \(y(5) = 0.5 + 5 = 5.5\)

Answer: \(C = 0.5\), \(D = 1\); predicted \(y(5) = 5.5\)

4.50. Solving a Linear System with Free Variables (Midterm Recap, Task 3)

Solve the system: \[\begin{bmatrix} 1 & 3 & 0 & 2 \\ 0 & 0 & 1 & 4 \\ 1 & 3 & 1 & 6 \end{bmatrix} \begin{bmatrix} x \\ y \\ z \\ w \end{bmatrix} = \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}\]

Click to see the solution

Key Concept: Row-reduce the augmented matrix to echelon form, identify pivot and free variables, then write the general solution.

1. Row reduce the augmented matrix (subtract row 1 from row 3): \[\left[\begin{array}{cccc|c} 1 & 3 & 0 & 2 & 1 \\ 0 & 0 & 1 & 4 & 2 \\ 0 & 0 & 0 & 0 & 0 \end{array}\right]\]
2. Identify variables: Pivot variables: \(x, z\). Free variables: \(y, w \in \mathbb{R}\).
3. Express pivot variables: \[\begin{cases} x = 1 - 3y - 2w \\ z = 2 - 4w \end{cases}\]
4. Write general solution: \[\begin{bmatrix} x \\ y \\ z \\ w \end{bmatrix} = \begin{bmatrix} 1 \\ 0 \\ 2 \\ 0 \end{bmatrix} + y\begin{bmatrix} -3 \\ 1 \\ 0 \\ 0 \end{bmatrix} + w\begin{bmatrix} -2 \\ 0 \\ -4 \\ 1 \end{bmatrix}, \quad y, w \in \mathbb{R}\]

Answer: The general solution is a particular solution plus two free directions as shown above.

4.51. Eigenvalues and Eigenvectors of a Tridiagonal Matrix (Midterm Recap, Task 4)

Find the eigenvalues and eigenvectors of: \[A = \begin{bmatrix} 4 & 1 & 0 \\ 1 & 4 & 1 \\ 0 & 1 & 4 \end{bmatrix}\]

Click to see the solution

1. Characteristic equation \(\det(A - \lambda I) = 0\): \[(4-\lambda)\left[(4-\lambda)^2 - 1\right] - (4-\lambda) = (4-\lambda)\left[(4-\lambda)^2 - 2\right] = 0\]

2. Eigenvalues:

   • \(4 - \lambda = 0 \Rightarrow \lambda_1 = 4\)
   • \((4-\lambda)^2 = 2 \Rightarrow \lambda_{2,3} = 4 \pm \sqrt{2}\)

   Spectrum: \(\Lambda = \{4,\ 4+\sqrt{2},\ 4-\sqrt{2}\}\)

3. Eigenvectors via \(Null(A - \lambda_i I)\):

   For \(\lambda_1 = 4\): \[\begin{cases} y = 0 \\ x + z = 0 \end{cases} \Rightarrow \vec{x}_1 = \alpha\begin{bmatrix} -1 \\ 0 \\ 1 \end{bmatrix}\]

   For \(\lambda_2 = 4 + \sqrt{2}\): \[\begin{cases} y = \sqrt{2}x \\ x = z \end{cases} \Rightarrow \vec{x}_2 = \beta\begin{bmatrix} 1/\sqrt{2} \\ 1 \\ 1/\sqrt{2} \end{bmatrix}\]

   For \(\lambda_3 = 4 - \sqrt{2}\): \[\vec{x}_3 = \gamma\begin{bmatrix} -1/\sqrt{2} \\ 1 \\ -1/\sqrt{2} \end{bmatrix}\]

Answer: Eigenvalues \(4,\ 4\pm\sqrt{2}\) with eigenvectors as above.

4.52. Fundamental Subspaces of a Rank-1 Matrix (Midterm Recap, Task 5)

Given \(A = \begin{bmatrix} 1 & 2 & 0 \\ 2 & 4 & 0 \\ 0 & 0 & 0 \end{bmatrix}\):

(a) Find \(Null(A)\) and \(Col(A)\).

(b) Find \(Null(A^T)\) and \(Col(A^T)\).

(c) Verify orthogonality of the fundamental subspaces.

Click to see the solution

Key Concept: The four fundamental subspaces of \(A\) satisfy: \(Null(A) \perp Col(A^T)\) and \(Null(A^T) \perp Col(A)\).

(a) Row-reduce \(A\): \(\begin{bmatrix} 1 & 2 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}\). Only one pivot: \(x + 2y = 0 \Rightarrow x = -2y\).

\[Null(A) = \text{span}\left\{ \begin{bmatrix} -2 \\ 1 \\ 0 \end{bmatrix},\ \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} \right\}\]

Consistency condition for \(Ax = b\): \(b_2 = 2b_1\) and \(b_3 = 0\), so: \[Col(A) = \text{span}\left\{ \begin{bmatrix} 1 \\ 2 \\ 0 \end{bmatrix} \right\}\]

(b) Since \(A\) is symmetric (\(A = A^T\)): \[Null(A^T) = Null(A), \quad Col(A^T) = Col(A)\]

(c) Check \(Null(A) \perp Col(A)\): \[[-2,\ 1,\ 0]\begin{bmatrix} 1 \\ 2 \\ 0 \end{bmatrix} = -2 + 2 + 0 = 0 \checkmark\] \[[0,\ 0,\ 1]\begin{bmatrix} 1 \\ 2 \\ 0 \end{bmatrix} = 0 \checkmark\]

Answer: Orthogonality holds for all pairs of fundamental subspaces.

4.53. Complex Matrix and Hermitian Conjugate (Midterm Recap, Task 6)

Given \(A = \begin{bmatrix} 1 & i \\ i & -1 \end{bmatrix}\) and \(b = \begin{bmatrix} 1 \\ i \end{bmatrix}\):

(a) Solve \(A\vec{x} = \vec{b}\) for \(\vec{x} \in \mathbb{C}^2\).

(b) Find \(Null(A)\), \(Col(A)\), \(Null(A^+)\), \(Col(A^+)\), where \(A^+ = \bar{A}^T\) is the Hermitian conjugate.

(c) Verify orthogonality using the complex inner product \(x^* y = \sum_j x_j \overline{y_j}\).

Click to see the solution

Key Concept: For complex matrices, orthogonality uses the Hermitian inner product: \(\langle u, v \rangle = u^+ v = \sum_j \bar{u}_j v_j\).

(a) The system: \[\begin{cases} x_1 + x_2 i = 1 \\ i x_1 - x_2 = i \end{cases}\]

Substituting the first equation into the second yields an identity \(i = i\), so \(x_2 \in \mathbb{C}\) is free: \[\vec{x} = \begin{bmatrix} 1 \\ 0 \end{bmatrix} + x_2\begin{bmatrix} -i \\ 1 \end{bmatrix}, \quad x_2 \in \mathbb{C}\]

(b)

\(Null(A)\): \(x_1 + x_2 i = 0 \Rightarrow x_1 = -x_2 i\), so \(Null(A) = \text{span}\left\{\begin{bmatrix} -i \\ 1 \end{bmatrix}\right\}\)

\(Col(A) = \text{span}\left\{\begin{bmatrix} 1 \\ i \end{bmatrix}\right\}\)

Hermitian conjugate: \(A^+ = \begin{bmatrix} 1 & -i \\ -i & -1 \end{bmatrix}\)

\(Null(A^+)\): \(x_1 - x_2 i = 0 \Rightarrow x_1 = x_2 i\), so \(Null(A^+) = \text{span}\left\{\begin{bmatrix} i \\ 1 \end{bmatrix}\right\}\)

\(Col(A^+) = \text{span}\left\{\begin{bmatrix} 1 \\ -i \end{bmatrix}\right\}\)

(c) Check \(Null(A^+) \perp Col(A)\): \[\begin{bmatrix} i \\ 1 \end{bmatrix}^+ \begin{bmatrix} 1 \\ i \end{bmatrix} = \bar{i}\cdot 1 + \bar{1}\cdot i = (-i)(1) + (1)(i) = -i + i = 0 \checkmark\]

Check \(Null(A) \perp Col(A^+)\): \[\begin{bmatrix} -i \\ 1 \end{bmatrix}^+ \begin{bmatrix} 1 \\ -i \end{bmatrix} = \overline{(-i)}\cdot 1 + \bar{1}\cdot(-i) = i - i = 0 \checkmark\]

Answer: Orthogonality holds in both pairs.

4.54. Inhomogeneous System and Null Space (Midterm Recap, Task 7)

Solve the inhomogeneous system and find the null space of the coefficient matrix: \[\begin{bmatrix} 1 & -1 & 2 & 0 \\ 2 & -2 & 4 & 1 \\ 3 & -3 & 6 & 1 \end{bmatrix} \begin{bmatrix} x \\ y \\ z \\ w \end{bmatrix} = \begin{bmatrix} 1 \\ 3 \\ 4 \end{bmatrix}\]

Click to see the solution

Key Concept: The general solution to \(Ax = b\) is \(x = x_p + x_h\), where \(x_p\) is any particular solution and \(x_h \in Null(A)\).

1. Row reduce the augmented matrix: \[\left[\begin{array}{cccc|c} 1 & -1 & 2 & 0 & 1 \\ 2 & -2 & 4 & 1 & 3 \\ 3 & -3 & 6 & 1 & 4 \end{array}\right] \sim \left[\begin{array}{cccc|c} 1 & -1 & 2 & 0 & 1 \\ 0 & 0 & 0 & 1 & 1 \\ 0 & 0 & 0 & 0 & 0 \end{array}\right]\]
2. Identify variables: Pivots: \(x, w\). Free: \(y, z \in \mathbb{R}\).
3. Express pivot variables: \[\begin{cases} x = 1 + y - 2z \\ w = 1 \end{cases}\]
4. General solution: \[\begin{bmatrix} x \\ y \\ z \\ w \end{bmatrix} = \underbrace{\begin{bmatrix} 1 \\ 0 \\ 0 \\ 1 \end{bmatrix}}_{x_p} + y\begin{bmatrix} 1 \\ 1 \\ 0 \\ 0 \end{bmatrix} + z\begin{bmatrix} -2 \\ 0 \\ 1 \\ 0 \end{bmatrix}, \quad y, z \in \mathbb{R}\]

Null space: \[Null(A) = \text{span}\left\{ \begin{bmatrix} 1 \\ 1 \\ 0 \\ 0 \end{bmatrix},\ \begin{bmatrix} -2 \\ 0 \\ 1 \\ 0 \end{bmatrix} \right\}\]

4.55. Four Fundamental Subspaces (Midterm 2026, Task 1)

Let \[A = \begin{pmatrix} 4 & 5 & -2 & 6 & 0 \\ 1 & 1 & 0 & 1 & 0 \end{pmatrix}.\]

(a) Find the four fundamental subspaces of \(A\).

(b) Verify the orthogonality relations between the corresponding subspaces.

Click to see the solution

Key Concept: The four fundamental subspaces are \(Col(A)\), \(Null(A)\), \(Row(A) = Col(A^T)\), and \(Null(A^T)\). They satisfy: \(Row(A) \perp Null(A)\) and \(Col(A) \perp Null(A^T)\).

(a) Row reduce \(A\):

\[A = \begin{pmatrix} 4 & 5 & -2 & 6 & 0 \\ 1 & 1 & 0 & 1 & 0 \end{pmatrix}\]

Swap rows (or use \(R_1 \leftarrow R_1 - 4R_2\)):

\[\sim \begin{pmatrix} 1 & 1 & 0 & 1 & 0 \\ 4 & 5 & -2 & 6 & 0 \end{pmatrix} \xrightarrow{R_2 - 4R_1} \begin{pmatrix} 1 & 1 & 0 & 1 & 0 \\ 0 & 1 & -2 & 2 & 0 \end{pmatrix} \xrightarrow{R_1 - R_2} \begin{pmatrix} 1 & 0 & 2 & -1 & 0 \\ 0 & 1 & -2 & 2 & 0 \end{pmatrix}\]

Pivot columns: 1 and 2. Free variables: \(x_3, x_4, x_5\).

\(Col(A)\) — column space: spanned by pivot columns of the original \(A\): \[Col(A) = \text{span}\left\{ \begin{pmatrix} 4 \\ 1 \end{pmatrix},\ \begin{pmatrix} 5 \\ 1 \end{pmatrix} \right\} = \mathbb{R}^2\]

Since \(A\) has 2 pivot rows, \(\text{rank}(A) = 2\), so \(Col(A) = \mathbb{R}^2\).

\(Null(A)\) — null space: from RREF, express pivot variables: \[x_1 = -2x_3 + x_4, \quad x_2 = 2x_3 - 2x_4\]

Setting \((x_3, x_4, x_5) = (1,0,0),\ (0,1,0),\ (0,0,1)\):

\[Null(A) = \text{span}\left\{ \begin{pmatrix} -2 \\ 2 \\ 1 \\ 0 \\ 0 \end{pmatrix},\ \begin{pmatrix} 1 \\ -2 \\ 0 \\ 1 \\ 0 \end{pmatrix},\ \begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \\ 1 \end{pmatrix} \right\}\]

\(Row(A)\) — row space: spanned by the nonzero rows of RREF: \[Row(A) = \text{span}\left\{ \begin{pmatrix} 1 \\ 0 \\ 2 \\ -1 \\ 0 \end{pmatrix},\ \begin{pmatrix} 0 \\ 1 \\ -2 \\ 2 \\ 0 \end{pmatrix} \right\}\]

\(Null(A^T)\) — left null space: \(A^T y = 0\) where \(A\) is \(2\times 5\), so \(A^T\) is \(5\times 2\). We need \(y \in \mathbb{R}^2\) such that \(A^T y = 0\), i.e., \(y^T A = 0\). Since \(\text{rank}(A) = 2 = m\), the matrix \(A\) has full row rank, so \(Null(A^T) = \{0\}\).

(b) Verify orthogonality:

\(Row(A) \perp Null(A)\): Check each basis vector of \(Null(A)\) against basis vectors of \(Row(A)\):

\[\begin{pmatrix} 1 \\ 0 \\ 2 \\ -1 \\ 0 \end{pmatrix}^T \begin{pmatrix} -2 \\ 2 \\ 1 \\ 0 \\ 0 \end{pmatrix} = -2 + 0 + 2 + 0 + 0 = 0 \checkmark\]

\[\begin{pmatrix} 1 \\ 0 \\ 2 \\ -1 \\ 0 \end{pmatrix}^T \begin{pmatrix} 1 \\ -2 \\ 0 \\ 1 \\ 0 \end{pmatrix} = 1 + 0 + 0 - 1 + 0 = 0 \checkmark\]

\[\begin{pmatrix} 1 \\ 0 \\ 2 \\ -1 \\ 0 \end{pmatrix}^T \begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \\ 1 \end{pmatrix} = 0 \checkmark\]

\[\begin{pmatrix} 0 \\ 1 \\ -2 \\ 2 \\ 0 \end{pmatrix}^T \begin{pmatrix} -2 \\ 2 \\ 1 \\ 0 \\ 0 \end{pmatrix} = 0 + 2 - 2 + 0 + 0 = 0 \checkmark\]

\[\begin{pmatrix} 0 \\ 1 \\ -2 \\ 2 \\ 0 \end{pmatrix}^T \begin{pmatrix} 1 \\ -2 \\ 0 \\ 1 \\ 0 \end{pmatrix} = 0 - 2 + 0 + 2 + 0 = 0 \checkmark\]

\[\begin{pmatrix} 0 \\ 1 \\ -2 \\ 2 \\ 0 \end{pmatrix}^T \begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \\ 1 \end{pmatrix} = 0 \checkmark\]

\(Col(A) \perp Null(A^T)\): Since \(Null(A^T) = \{0\}\), orthogonality holds trivially.

Answer: \(Col(A) = \mathbb{R}^2\); \(Null(A) = \text{span}\left\{\begin{pmatrix}-2\\2\\1\\0\\0\end{pmatrix}, \begin{pmatrix}1\\-2\\0\\1\\0\end{pmatrix}, \begin{pmatrix}0\\0\\0\\0\\1\end{pmatrix}\right\}\); \(Row(A) = \text{span}\left\{\begin{pmatrix}1\\0\\2\\-1\\0\end{pmatrix}, \begin{pmatrix}0\\1\\-2\\2\\0\end{pmatrix}\right\}\); \(Null(A^T) = \{0\}\). Orthogonality verified.

4.56. Least-Squares Line via Normal Equations (Midterm 2026, Task 2)

Use the normal equations to find the least-squares line \(y = ax + b\) for the points \[(1,2),\ (2,4),\ (3,5),\ (4,4),\ (5,6).\]

Click to see the solution

Key Concept: Set up the overdetermined system \(X\beta = y\) where \(X\) contains a column of \(x\)-values and a column of ones. Solve the normal equations \(X^T X \hat{\beta} = X^T y\).

1. Set up the matrix \(X\) and vector \(y\): \[X = \begin{pmatrix} 1 & 1 \\ 2 & 1 \\ 3 & 1 \\ 4 & 1 \\ 5 & 1 \end{pmatrix}, \quad y = \begin{pmatrix} 2 \\ 4 \\ 5 \\ 4 \\ 6 \end{pmatrix}\]

2. Compute \(X^T X\): \[X^T X = \begin{pmatrix} 1^2+2^2+3^2+4^2+5^2 & 1+2+3+4+5 \\ 1+2+3+4+5 & 5 \end{pmatrix} = \begin{pmatrix} 55 & 15 \\ 15 & 5 \end{pmatrix}\]

3. Compute \(X^T y\): \[X^T y = \begin{pmatrix} 1\cdot2+2\cdot4+3\cdot5+4\cdot4+5\cdot6 \\ 2+4+5+4+6 \end{pmatrix} = \begin{pmatrix} 2+8+15+16+30 \\ 21 \end{pmatrix} = \begin{pmatrix} 71 \\ 21 \end{pmatrix}\]

4. Solve the normal equations \(\begin{pmatrix}55 & 15\\15 & 5\end{pmatrix}\begin{pmatrix}a\\b\end{pmatrix} = \begin{pmatrix}71\\21\end{pmatrix}\):

   From the second equation: \(15a + 5b = 21 \Rightarrow b = \frac{21 - 15a}{5}\).

   Substitute into the first: \(55a + 15 \cdot \frac{21-15a}{5} = 71\) \[55a + 3(21-15a) = 71 \Rightarrow 55a + 63 - 45a = 71 \Rightarrow 10a = 8 \Rightarrow a = 0.8\] \[b = \frac{21 - 15(0.8)}{5} = \frac{21 - 12}{5} = \frac{9}{5} = 1.8\]

Answer: The least-squares line is \(y = 0.8x + 1.8\).

4.57. Eigenvalues, Eigenvectors, and Matrix Power (Midterm 2026, Task 3)

Let \[A = \begin{pmatrix} 4 & 2 \\ 2 & 4 \end{pmatrix}.\]

(a) Find the eigenvalues and eigenvectors of \(A\).

(b) Compute \(A^3 - A\).

(c) Verify that the eigenvectors are orthogonal.

Click to see the solution

Key Concept: For a symmetric matrix, eigenvalues are real and eigenvectors for distinct eigenvalues are orthogonal. Use diagonalization \(A = PDP^{-1}\) to compute matrix powers.

(a) Eigenvalues: Solve \(\det(A - \lambda I) = 0\): \[(4-\lambda)^2 - 4 = 0 \Rightarrow \lambda^2 - 8\lambda + 12 = 0 \Rightarrow (\lambda-6)(\lambda-2) = 0\] \[\lambda_1 = 6, \quad \lambda_2 = 2\]

Eigenvectors:

For \(\lambda_1 = 6\): \((A - 6I)v = 0\): \[\begin{pmatrix} -2 & 2 \\ 2 & -2 \end{pmatrix} \rightarrow v_1 = \begin{pmatrix} 1 \\ 1 \end{pmatrix}\]

For \(\lambda_2 = 2\): \((A - 2I)v = 0\): \[\begin{pmatrix} 2 & 2 \\ 2 & 2 \end{pmatrix} \rightarrow v_2 = \begin{pmatrix} 1 \\ -1 \end{pmatrix}\]

(b) Compute \(A^3 - A\):

Using diagonalization \(A = PDP^{-1}\), we have \(A^k = PD^kP^{-1}\), so: \[A^3 - A = P(D^3 - D)P^{-1}\]

\[D^3 - D = \begin{pmatrix} 6^3 - 6 & 0 \\ 0 & 2^3 - 2 \end{pmatrix} = \begin{pmatrix} 210 & 0 \\ 0 & 6 \end{pmatrix}\]

With \(P = \begin{pmatrix}1 & 1\\ 1 & -1\end{pmatrix}\), \(P^{-1} = \frac{1}{2}\begin{pmatrix}1 & 1\\ 1 & -1\end{pmatrix}\):

\[A^3 - A = \frac{1}{2}\begin{pmatrix}1 & 1\\1 & -1\end{pmatrix}\begin{pmatrix}210 & 0\\0 & 6\end{pmatrix}\begin{pmatrix}1 & 1\\1 & -1\end{pmatrix}\]

\[= \frac{1}{2}\begin{pmatrix}210 & 6\\210 & -6\end{pmatrix}\begin{pmatrix}1 & 1\\1 & -1\end{pmatrix} = \frac{1}{2}\begin{pmatrix}216 & 204\\204 & 216\end{pmatrix} = \begin{pmatrix}108 & 102\\102 & 108\end{pmatrix}\]

(c) Verify orthogonality: \[v_1 \cdot v_2 = \begin{pmatrix}1\\1\end{pmatrix}^T\begin{pmatrix}1\\-1\end{pmatrix} = 1\cdot1 + 1\cdot(-1) = 0 \checkmark\]

Answer: (a) \(\lambda_1 = 6\), \(v_1 = \begin{pmatrix}1\\1\end{pmatrix}\); \(\lambda_2 = 2\), \(v_2 = \begin{pmatrix}1\\-1\end{pmatrix}\). (b) \(A^3 - A = \begin{pmatrix}108 & 102\\102 & 108\end{pmatrix}\). (c) \(v_1 \cdot v_2 = 0\) — orthogonal.

4.58. Solving a Linear System (Midterm 2026, Task 4)

Solve the system \[\begin{cases} x_1 + 2x_2 + 3x_3 + 4x_4 = 5, \\ 2x_1 + 4x_2 + 6x_3 + 8x_4 = 10, \\ x_1 + 2x_2 + 2x_3 + 3x_4 = 4. \end{cases}\]

Click to see the solution

Key Concept: Row reduce the augmented matrix to find pivot and free variables, then write the general solution as a particular solution plus the null space.

1. Form and row-reduce the augmented matrix: \[\left[\begin{array}{cccc|c} 1 & 2 & 3 & 4 & 5 \\ 2 & 4 & 6 & 8 & 10 \\ 1 & 2 & 2 & 3 & 4 \end{array}\right]\]

   \(R_2 \leftarrow R_2 - 2R_1\), \(R_3 \leftarrow R_3 - R_1\): \[\sim \left[\begin{array}{cccc|c} 1 & 2 & 3 & 4 & 5 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & -1 & -1 & -1 \end{array}\right]\]

   Swap \(R_2 \leftrightarrow R_3\), then \(R_2 \leftarrow -R_2\): \[\sim \left[\begin{array}{cccc|c} 1 & 2 & 3 & 4 & 5 \\ 0 & 0 & 1 & 1 & 1 \\ 0 & 0 & 0 & 0 & 0 \end{array}\right]\]

   \(R_1 \leftarrow R_1 - 3R_2\): \[\sim \left[\begin{array}{cccc|c} 1 & 2 & 0 & 1 & 2 \\ 0 & 0 & 1 & 1 & 1 \\ 0 & 0 & 0 & 0 & 0 \end{array}\right]\]

2. Identify variables: Pivots: \(x_1, x_3\). Free: \(x_2 = s\), \(x_4 = t\).

3. Express pivot variables: \[x_3 = 1 - t, \quad x_1 = 2 - 2s - t\]

4. General solution: \[\begin{pmatrix}x_1\\x_2\\x_3\\x_4\end{pmatrix} = \begin{pmatrix}2\\0\\1\\0\end{pmatrix} + s\begin{pmatrix}-2\\1\\0\\0\end{pmatrix} + t\begin{pmatrix}-1\\0\\-1\\1\end{pmatrix}, \quad s,t \in \mathbb{R}\]

Answer: \(x = \begin{pmatrix}2\\0\\1\\0\end{pmatrix} + s\begin{pmatrix}-2\\1\\0\\0\end{pmatrix} + t\begin{pmatrix}-1\\0\\-1\\1\end{pmatrix}\), \(s,t \in \mathbb{R}\).

4.59. Projection Matrix and Projection of a Vector (Midterm 2026, Task 5)

Let \[A = \begin{pmatrix} 1 & 1 \\ 0 & 1 \\ 1 & 0 \end{pmatrix}, \quad b = \begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix}.\]

(a) Find the projection matrix \(P\) onto \(\text{Col}(A)\).

(b) Find the projection of \(b\) onto \(\text{Col}(A)\).

Click to see the solution

Key Concept: The projection matrix onto \(Col(A)\) is \(P = A(A^T A)^{-1}A^T\). The projection of \(b\) is \(\hat{b} = Pb\).

(a) Compute \(A^T A\): \[A^T A = \begin{pmatrix}1&0&1\\1&1&0\end{pmatrix}\begin{pmatrix}1&1\\0&1\\1&0\end{pmatrix} = \begin{pmatrix}1+0+1 & 1+0+0\\ 1+0+0 & 1+1+0\end{pmatrix} = \begin{pmatrix}2 & 1\\ 1 & 2\end{pmatrix}\]

Compute \((A^T A)^{-1}\): \[\det(A^T A) = 4 - 1 = 3, \quad (A^T A)^{-1} = \frac{1}{3}\begin{pmatrix}2 & -1\\ -1 & 2\end{pmatrix}\]

Compute \(A^T\): \[A^T = \begin{pmatrix}1&0&1\\1&1&0\end{pmatrix}\]

Compute \(P = A(A^T A)^{-1}A^T\):

First, \(A(A^T A)^{-1} = \frac{1}{3}\begin{pmatrix}1&1\\0&1\\1&0\end{pmatrix}\begin{pmatrix}2&-1\\-1&2\end{pmatrix} = \frac{1}{3}\begin{pmatrix}1&1\\-1&2\\2&-1\end{pmatrix}\)

Then: \[P = \frac{1}{3}\begin{pmatrix}1&1\\-1&2\\2&-1\end{pmatrix}\begin{pmatrix}1&0&1\\1&1&0\end{pmatrix} = \frac{1}{3}\begin{pmatrix}2&1&1\\1&2&-1\\1&-1&2\end{pmatrix}\]

(b) Compute \(\hat{b} = Pb\): \[\hat{b} = \frac{1}{3}\begin{pmatrix}2&1&1\\1&2&-1\\1&-1&2\end{pmatrix}\begin{pmatrix}1\\1\\1\end{pmatrix} = \frac{1}{3}\begin{pmatrix}4\\2\\2\end{pmatrix} = \begin{pmatrix}4/3\\2/3\\2/3\end{pmatrix}\]

Answer: (a) \(P = \frac{1}{3}\begin{pmatrix}2&1&1\\1&2&-1\\1&-1&2\end{pmatrix}\). (b) \(\hat{b} = \begin{pmatrix}4/3\\2/3\\2/3\end{pmatrix}\).

4.60. Complex Matrix: Null Space and Orthogonality (Midterm 2026, Task 6)

Let \[A = \begin{pmatrix} i & 1 & i \\ 1 & -i & 0 \end{pmatrix}.\]

(a) Solve \(Ax = 0\).

(b) Show that \(N(A) \perp C(A^H)\), but \(N(A) \not\perp C(A^T)\).

Click to see the solution

Key Concept: The fundamental theorem of linear algebra for complex matrices states \(N(A) \perp C(A^H)\) (orthogonality uses the Hermitian inner product \(\langle u,v \rangle = u^H v\)). This does NOT hold for \(A^T\) because the transpose does not conjugate.

(a) Solve \(Ax = 0\):

Row reduce \(A\) over \(\mathbb{C}\). Multiply \(R_1\) by \(-i\) (i.e., \(R_1 \leftarrow -i\cdot R_1\)): \[R_1 \leftarrow -i\cdot R_1: \quad \begin{pmatrix} 1 & -i & 1 \\ 1 & -i & 0 \end{pmatrix}\]

\(R_2 \leftarrow R_2 - R_1\): \[\begin{pmatrix} 1 & -i & 1 \\ 0 & 0 & -1 \end{pmatrix}\]

\(R_1 \leftarrow R_1 + R_2\): \[\begin{pmatrix} 1 & -i & 0 \\ 0 & 0 & -1 \end{pmatrix} \xrightarrow{R_2 \leftarrow -R_2} \begin{pmatrix} 1 & -i & 0 \\ 0 & 0 & 1 \end{pmatrix}\]

Pivots: \(x_1, x_3\). Free: \(x_2 = t\).

From the RREF: \(x_3 = 0\), \(x_1 = ix_2 = it\).

\[N(A) = \text{span}\left\{ \begin{pmatrix} i \\ 1 \\ 0 \end{pmatrix} \right\}\]

(b) \(N(A) \perp C(A^H)\):

Compute \(A^H\) (conjugate transpose of \(A\)): \[A^H = \begin{pmatrix} -i & 1 \\ 1 & i \\ -i & 0 \end{pmatrix}\]

\(C(A^H) = \text{span}\left\{ \begin{pmatrix}-i\\1\\-i\end{pmatrix},\ \begin{pmatrix}1\\i\\0\end{pmatrix} \right\}\)

Let \(v = \begin{pmatrix}i\\1\\0\end{pmatrix}\). Check \(\langle v, a_1^H \rangle = v^H a_1^H\): \[v^H = \begin{pmatrix}-i & 1 & 0\end{pmatrix}, \quad v^H \begin{pmatrix}-i\\1\\-i\end{pmatrix} = (-i)(-i) + (1)(1) + 0 = i^2\cdot(-1)\cdot(-1)\]

More carefully: \((-i)(-i) = i^2 = -1\), so: \[(-i)(-i) + 1\cdot1 + 0 = -1 + 1 = 0 \checkmark\]

\[v^H \begin{pmatrix}1\\i\\0\end{pmatrix} = (-i)(1) + (1)(i) + 0 = -i + i = 0 \checkmark\]

So \(N(A) \perp C(A^H)\). ✓

\(N(A) \not\perp C(A^T)\):

Compute \(A^T\) (transpose without conjugation): \[A^T = \begin{pmatrix} i & 1 \\ 1 & -i \\ i & 0 \end{pmatrix}\]

\(C(A^T) = \text{span}\left\{\begin{pmatrix}i\\1\\i\end{pmatrix},\ \begin{pmatrix}1\\-i\\0\end{pmatrix}\right\}\)

The correct inner product on \(\mathbb{C}^n\) is the Hermitian one \(u^H v\). Check with \(v = \begin{pmatrix}i\\1\\0\end{pmatrix}\) against columns of \(A^T\):

\[v^H \begin{pmatrix}i\\1\\i\end{pmatrix} = \overline{i}\cdot i + \overline{1}\cdot 1 + \overline{0}\cdot i = (-i)(i) + 1 + 0 = -i^2 + 1 = 1 + 1 = 2 \neq 0\]

So \(N(A) \not\perp C(A^T)\) under the standard Hermitian inner product. ✗

Answer: (a) \(N(A) = \text{span}\left\{\begin{pmatrix}i\\1\\0\end{pmatrix}\right\}\). (b) \(N(A) \perp C(A^H)\) since \(v^H(A^H\text{ columns}) = 0\); \(N(A) \not\perp C(A^T)\) since \(v^H\begin{pmatrix}i\\1\\i\end{pmatrix} = 2 \neq 0\).